Codespace (custom docker) no longer loads in JupyterLab

[eculler]

JupyterLab Codespaces

I have been configuring a codespace with a docker image (which has jupyterlab installed in a conda environment which is activated by the Dockerfile). Today, I am suddenly unable to access it in JupyterLab.

My codespace is still accessible in the browser version of VSCode. However, when I open in in JupyterLab, I get a white screen with a black border. Before the blank screen, it displays the "Setting up your Codespace" window, sometimes following by the "Connecting" screen and sometimes not. Normally at this point the JupyterLab logo would appear and then I would be able to use JupyterLab.

Often but not always when I look at the developer tools I get a 500 Internal Server Error in response to a GET request to mycodespace.preview.app.github.dev/lab/?token=...

I also always get the warnings "Content Security Policy: Couldn’t process unknown directive ‘require-trusted-types-for’" and "Layout was forced before the page was fully loaded. If stylesheets are not yet loaded this may cause a flash of unstyled content."

In VSCode port 8888 is visible and running. When I try opening the url for that port directly and insert the correct token I get another 500 Internal Server Error. This time, the response says "Jupyter Server requires JavaScript. Please enable it to proceed."

I have tried reverting the docker image and the .devcontainer.json to a version I know was working last week, multiple browsers (Firefox, Chrome, Safari) and deleting my browser cache, all with no change.

I looked at issue #50757, but it was not solved (and I have verified that /opt/conda/bin is in the codespace PATH as suggested there).

[eljog]

Thank you for reporting this issue. Our engineering team will look into it and get back

[melund]

I have run into the same issue. It is an image build on top of the default codespace image: mcr.microsoft.com/devcontainers/universal

But as soon as I add the devcontainer feature which installs conda/mamba it fails in the exact same way as @eculler outlines.

The following makes it easy to reproduce.

devcontainer.json

image: "mcr.microsoft.com/devcontainers/universal",
"features": {
    "ghcr.io/mamba-org/devcontainer-features/micromamba:1": {
        "channels": "conda-forge",
        "packages": "python>=3.11 chardet tqdm ipywidgets ipykerneljupyterlab jupyterlab-git"
     }
}

@eljog; I wonder if the codespaces uses any special tricks when starting the jupyter server? Do we miss certain packages in the conda environment?

[four43]

Fix TL;DR - Ensure you can run jupyter lab from the command line. I was running as root and it wanted an extra flag, even though from within VSCode it would start a kernel fine. I am still getting an issue connecting with the token, however. A 502 from https://[codespace].app.github.dev/lab/?token=[token]

Main Testing

I am having the same issue. What is Codespaces doing to start the server? Do I need to forward some ports? I'm using a docker-compose setup.

I can start the Codespace, connect with Web and local VSCode, I can open jupyter notebooks and the local server can starts up and executes.

[Live edits as I debug, thanks for being a good notebook, Github :D]
From within the running Codespace terminal:

$ pip show jupyterlab
Name: jupyterlab
Version: 4.1.2
Summary: JupyterLab computational environment
...

$ which jupyter
/usr/local/bin/jupyter

$ jupyter lab
[I 2024-03-27 15:11:38.152 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-03-27 15:11:38.154 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-03-27 15:11:38.156 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-03-27 15:11:38.156 ServerApp] Writing Jupyter server cookie secret to /root/.local/share/jupyter/runtime/jupyter_cookie_secret
[I 2024-03-27 15:11:38.322 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-03-27 15:11:38.360 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-03-27 15:11:38.361 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-03-27 15:11:38.361 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-03-27 15:11:38.363 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.11/site-packages/jupyterlab
[I 2024-03-27 15:11:38.363 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab
[I 2024-03-27 15:11:38.364 LabApp] Extension Manager is 'pypi'.
[I 2024-03-27 15:11:38.370 ServerApp] jupyterlab | extension was successfully loaded.
[C 2024-03-27 15:11:38.371 ServerApp] Running as root is not recommended. Use --allow-root to bypass.

$ echo $?
1

🤔

[Run devcontainer as a new, unprivileged user]

$ ps -ef |  grep jupyter-lab
app-user    3455       1  1 16:01 ?        00:00:01 /usr/local/bin/python3.11 /usr/local/bin/jupyter-lab --ServerApp.allow_origin=* --ServerApp.cookie_options={'SameSite': 'None', 'Secure': True} --ServerApp.tornado_settings={'headers':{'Content-Security-Policy':"frame-ancestors 'self' https://[my-codespace].github.dev", 'Access-Control-Allow-Headers': 'accept, content-type, authorization, x-xsrftoken, x-github-token'}} --notebook-dir=/workspaces/[my-project] --no-browser

502 and 504s as I Open in JupyterLab, https://[codespace].app.github.dev/lab/?token=[token] It looks like a web socket fires up and has a lot of encrypted back and forth that maybe gets that token? Also there are some analytics going back and forth here. Maybe you guys can pull some logs from your tracking info :)