FYI, this is what the lock file looks like that it creates:

--w------- 1 nobody nogroup 6 Sep 11 13:14 /tmp/md.pid

Investigating further, this seems to have nothing to do with the user running the process. I've found that even when the pid file contains the correct process ID (as returned by ps it still complains it hasn't locked the file.

On 11-Sep-2017, Joshua Kugler wrote:

I have my program invoking the DaemonContext like so:
=20
with daemon.DaemonContext(umask=3D0x777, uid=3Duid, gid=3Dgid, detach_process=3D(not opts.run_in_foregroun= d), files_preserve=3D[h.stream for h in log.roo= t.handlers], pidfile=3Dmy_pidfile, stdout=3Dsys.stdout, = stderr=3Dsys.stderr, signal_map=3Dsignal_map): module.run()
uid/gid are the ids for the nobody/nogroup
my_pidfile is a TimeoutPIDLockFile object
=20
When that runs, I get a pid file at the correct place (/tmp/foo.pid in my=
case), owned by nobody/nogroup, but when the daemon tries to exit, I get t=
his traceback:
[=E2=80=A6] File "/usr/local/lib/python2.7/dist-packages/lockfile/pidlockfile.py", = line 105, in release raise NotMyLock("%s is locked, but not by me" % self.path) lockfile.NotMyLock: /tmp/md.pid is locked, but not by me Error in atexit._run_exitfuncs: [=E2=80=A6]
Any ideas?

Thank you for this report.

Looking at the code which handles the pidfile parameter, AFAICT it
definitely waits until it's in the daemon process (the process which
survives the fork) before using the PID file. So I don't immediately
see why you'd be getting =E2=80=9CNotMyLock=E2=80=9D.

Can you make a complete, simple example program to demonstrate this?
Ignore your own program, and just try to make a minimal program which
will demonstrate this behaviour so I can run it.

--=20
\ =E2=80=9CI like to reminisce with people I don't know. Granted,=
it |
`\ takes longer.=E2=80=9D =E2=80=94St=
even Wright |
_o__) |
Ben Finney ben@benfinney.id.au

Here we go. :) When running the program, I see this:

[jkugler:~/repos/newcontext/so_mvp_matcher] master+* 130 ± ls -l /tmp/loop.pid 
--w------- 1 jkugler jkugler 6 Sep 12 12:38 /tmp/loop.pid
[jkugler:~/repos/newcontext/so_mvp_matcher] master+* ± rm /tmp/loop.pid 
[jkugler:~/repos/newcontext/so_mvp_matcher] master+* ± sudo cat /tmp/loop.pid 
12108
[jkugler:~/repos/newcontext/so_mvp_matcher] master+* ± ps -AF|grep loop
jkugler  12108 12346  9 17817 35884   7 12:39 pts/1    00:00:01 /usr/bin/python matching_action/looper.py -f
jkugler  12170 26665  0  3703  1020   0 12:39 pts/3    00:00:00 grep --color=auto loop

The lock file is owned by the user running the process, and the PID matches. When when I hit ctrl-c (This is running not-detached), I get:

^CTraceback (most recent call last):
  File "matching_action/looper.py", line 55, in <module>
    main()
  File "matching_action/looper.py", line 52, in main
    loop.run()
  File "/usr/local/lib/python2.7/dist-packages/daemon/daemon.py", line 430, in __exit__
    self.close()
  File "/usr/local/lib/python2.7/dist-packages/daemon/daemon.py", line 424, in close
    self.pidfile.__exit__(None, None, None)
  File "/usr/local/lib/python2.7/dist-packages/lockfile/__init__.py", line 204, in __exit__
    self.release()
  File "/usr/local/lib/python2.7/dist-packages/lockfile/pidlockfile.py", line 105, in release
    raise NotMyLock("%s is locked, but not by me" % self.path)
lockfile.NotMyLock: /tmp/loop.pid is locked, but not by me
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/local/lib/python2.7/dist-packages/daemon/daemon.py", line 424, in close
    self.pidfile.__exit__(None, None, None)
  File "/usr/local/lib/python2.7/dist-packages/lockfile/__init__.py", line 204, in __exit__
    self.release()
  File "/usr/local/lib/python2.7/dist-packages/lockfile/pidlockfile.py", line 105, in release
    raise NotMyLock("%s is locked, but not by me" % self.path)
NotMyLock: /tmp/loop.pid is locked, but not by me
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/local/lib/python2.7/dist-packages/daemon/daemon.py", line 424, in close
    self.pidfile.__exit__(None, None, None)
  File "/usr/local/lib/python2.7/dist-packages/lockfile/__init__.py", line 204, in __exit__
    self.release()
  File "/usr/local/lib/python2.7/dist-packages/lockfile/pidlockfile.py", line 105, in release
    raise NotMyLock("%s is locked, but not by me" % self.path)
lockfile.NotMyLock: /tmp/loop.pid is locked, but not by me

lsof seems to indicate it does not keep the PID open.
Also, the utility lslocks indicates the lock is not active while the program is running.

Same situation when the program detaches.

At any case, here is my test case:

#!/usr/bin/python
import daemon
import math
import os
import signal
import sys
import time
from daemon import pidfile
class Loop(object):
    def __init__(self):
        self.end_loop = False
        self.ended = False
    def run(self):
        self.start_time = int(time.time())
        while not self.end_loop:
            print int(time.time()) - self.start_time
            cur_time = time.time()
            time.sleep(math.ceil(cur_time) - cur_time)
        print 'Loop exiting'
    def start(self):
        self.run()
    def stop(self):
        self.end_loop = True
        self.ended = True
def main():
    loop = Loop()
    signal_map = {signal.SIGTERM: loop.stop, signal.SIGINT: loop.stop}
    if len(sys.argv) > 1 and sys.argv[1] == '-f':
        detach_process = False
        print 'Not detaching'
    else:
        detach_process = True
        print 'Detaching'
    my_pidfile = pidfile.TimeoutPIDLockFile('/tmp/loop.pid')
    with daemon.DaemonContext(umask=0x777, uid=os.getuid(), gid=os.getgid(),
                              detach_process=detach_process,
                              pidfile=my_pidfile, stdout=sys.stdout, stderr=sys.stderr,
                              signal_map=signal_map):
        loop.run()
if __name__ == '__main__':
    main()

In case you're still stuck on this, or someone else stumbles across this, the problem is the umask value you're setting. 0x777 doesn't mean give all access, it means remove all access. To fix the problem, change the umask to something like 0o077 (It should be octal, not hex), then your example works for me.

I haven't dug deeply, but I suspect the owner write bit is being set because the file is being created/opened for write. The NotMyLock test at shutdown fails because the file is not readable by anyone except root. The pidfile code around reading the pidfile is wrapped by a try/except for IOError which silently drops the exception.

I wouldn't expect lslocks nor lsof to show the file. That's not the type of lock that a pid lockfile is. A pid lock file simply writes the current pid to a file. If another process tries to start and it's pid doesn't match and the pid within the file matches a running process, then it's considered locked. lslocks appears to be related to flock/fcntl/etc. The pidfile is not kept open, so lsof wouldn't show it.

Check the man page for umask for more details. In short, umask specifies the bits to turn off. Eg.

>>> 0o644 & ~0o777
0
>>> oct(0o644 & ~0o077)
'0600'

You're absolutely right. 0x is not what I should be doing. I should be using octal, and not hex...totally missed that. As to the actual umask, 077 does make a whole lot more sense.

Just tried it! Changing that to 0o077 fixed the issue. Thank you very much!

So, I suppose this issue could be closed. Seems it was PEBKAC all along... :)

21-Apr-2018, James O'Kane wrote:

In case you're still stuck on this, or someone else stumbles across
this, the problem is the umask value you're setting. 0x777 doesn't
mean give all access, it means remove all access.

Thank you. For future readers who may not know about umask values, the
formal definition is that “The file mode creation mask of the process
is used to turn off permission bits in the mode argument supplied
during calls to [file creation functions]”, according to the definition of umask.

To fix the problem, change the umask to something like 0o077 (It
should be octal, not hex), then your example works for me.

Yes, some may not be familiar with the convention that a umask is
expressed in octal notation (see the Unix Permissions Calculator tool), nor with the fact that the
umask is applied to restrict the specified permissions on future
system calls (see the Wikipedia article for umask).

On 21-Apr-2018, Joshua Kugler wrote:

You're absolutely right. 0x is not what I should be doing. I
should be using octal, and not hex...totally missed that. As to the
actual umask, 077 does make a whole lot more sense.

Just tried it! Changing that to 0o077 fixed the issue. Thank you
very much!

Thanks for confirming! I will close this issue.