Static FILES_DIRECTORY is not compatible with Kubernetes Deployments

[smueller18]

We use Chainlit as a dependency in a custom Python package that runs in a Kubernetes environment. The structure of the project is as follows:

APP_ROOT
 |- .chainlit/        (config_dir)
     |- config.toml
 |- .files/           (FILES_DIRECTORY)
 |- app/              (custom app)
     |- __init__.py
     |- app.py
 |- chainlit.md

As the FILES_DIRECTORY must be a writeable directory, a volume must be mounted inside the container at path APP_ROOT/.files. This works fine during starting and running, but on shutdown, Chainlit tries to remove the FILES_DIRECTORY recursively including the directory itself.

if self.files_dir.is_dir():
    shutil.rmtree(self.files_dir)

Calling this function results in an error message during shutdown:

ERROR:    Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 677, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.11/contextlib.py", line 211, in __aexit__
    await anext(self.gen)
  File "/usr/local/lib/python3.11/site-packages/chainlit/server.py", line 132, in lifespan
    shutil.rmtree(FILES_DIRECTORY)
  File "/usr/local/lib/python3.11/shutil.py", line 738, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/usr/local/lib/python3.11/shutil.py", line 736, in rmtree
    os.rmdir(path, dir_fd=dir_fd)
OSError: [Errno 30] Read-only file system: '/app/.files'
ERROR:    Application shutdown failed. Exiting.

As the FILES_DIRECTORY is hard coded at APP_ROOT/.files, there is no possibility to create a mount, where the .files directory is writable because it is no option to mount the whole APP_ROOT as it would overwrite all other required files.

At the moment we use a workaround by setting a custom workdir to a writable folder and then copy .chainlit and chainlit.md to this folder during container start.

In addition, we want to use a high availability setup, where multiple containers share the same volume. During shutdown of any pod, all files also from the other pods are deleted.

These two issues described can be fixed when the FILES_DIRECTORY can be customized for each runtime instance separately. The new folder structure could look like this:

APP_ROOT
 |- .chainlit/        (config_dir)
     |- config.toml
 |- app/              (custom app)
     |- __init__.py
     |- app.py
 |- chainlit.md
 |- .cache/          (volume mount)
     |- pod1/        (FILES_DIRECTORY for pod 1)
     |- pod2/        (FILES_DIRECTORY for pod 2)

So my proposal is to allow users to customize the files directory by an environment variable. If you agree with this solution, I can contribute this feature.

[cccc11231]

Question: is there a way to disable this feature? stop generating FILES_DIRECTORY