Hi everyone,
I have recently made an application, that utilizes LLMs to write code executed in Jupyter kernels.
An instance of a class that manages those jupyter sessions (KernelManager) was defined as a global variable, because if i initialize kernelManager twice I will loose already started sessions and they have to persist.
The problem comes because i need to optimize my application and it works well with multiple workers in my gunicorn server configuration. Unfortunately, they do not see existing kernel sessions, which I think is caused by using the KernelManager as a global variable.
I might give it a shot, serialize KernelManager instance, and store it in dcc. Store, but its a complex class and I do not think it’s going to work.
Do you have any suggestions on how I could deal with this situation?
Thank you for the support!