So, I have a application where I am using daemon to run a cheroot server as a daemon process.
And I found that my API does not work correctly if I am using python-daemon. I figured out that if I comment out close_all_open_files(exclude=exclude_fds) - everything works fine.
close_all_open_files(exclude=exclude_fds)
I was wondering what options I have here...
The reproducible example is also pretty complicated (And I am not able to reduce it further)
The issue is that if I do:
# Test 1 :::: Without python-daemon (I remove the DaemonContext) $ curl http://localhost:5000 Hello # Test 2 :::: With python-daemon - it shows me headers and so on in the BODY of my response $ curl http://localhost:5000 Test Log Message HTTP/1.1 200 OK Content-Type: text/plain Transfer-Encoding: chunked Date: Thu, 09 Mar 2023 15:52:04 GMT Server: Cheroot/9.0.0 5 Hello 0 # Test 3 :::: Commenting out `close_all_open_files(exclude=exclude_fds)` in python-daemon makes it work properly again $ curl http://localhost:5000 Hello
File: run.py
run.py
import os import signal import sys import logging from daemon import DaemonContext from cheroot.wsgi import WSGIServer PIDFILE = "./app.pid" # This run.py can be any file! - just opening a file causes the issue with open("./run.py") as f: # <--- Removing this makes it all work! import logging handler = logging.FileHandler('./app.log') logging.root.addHandler(handler) # <--- Removing this makes it all work! def app(environ, start_response): # WSGI application start_response("200 OK", [("Content-Type", "text/plain")]) logging.warning("Test Log Message") # <--- Removing this makes it all work! return [b"Hello"] def clean_existing_process(): # Kill older process if pidfile is present if os.path.exists(PIDFILE): with open(PIDFILE, "r") as pf: pid = int(pf.read().strip()) try: os.kill(pid, signal.SIGTERM) print(f"Killed process {pid}. Deleting file") except OSError as err: if "No such process" in str(err): print("pidfile exists, but process not running. Deleting file") else: print(f"Unable to stop process {pid}. Error: {err}") sys.exit(1) os.remove(PIDFILE) class Pidfile: def __enter__(self): pid = os.getpid() with open(PIDFILE, "w") as f: f.write(f"{pid}\n") def __exit__(self): if os.path.exists(PIDFILE): os.remove(PIDFILE) def run_server(): server = WSGIServer(("0.0.0.0", 5000), app) server.safe_start() def main(): clean_existing_process() print("Starting process...") with DaemonContext(pidfile=Pidfile(), working_directory=os.path.abspath(".")): run_server() if __name__ == "__main__": main()
What you have disabled, is a normal part of starting the daemon: Close all open file handles. This is so that the daemon process is properly detached from any context, unless specifically arranged otherwise.
Please see the DaemonContext docstring for all options you can use when creating the object, to customise behaviour as needed for your daemon process. This includes the option files_preserve.
files_preserve
Metadata Update from @bignose: - Issue close_status updated to: Invalid - Issue status updated to: Closed (was: Open)
Ah, I see.
Seems like the FileHandler was opening a file and that same fileno was being used by my API. Hence causing my streams mingled up.
I took a look at the files_preserve option, and looks like I can get it to work using that. Thanks !