Allow for other Python distributions in the browser

[jtpio]

Looking briefly at the code, it looks like the Web Worker current relies on Pyodide:

stlite/packages/kernel/src/worker.ts

Line 1 in 5a218ba

import { PyodideInterface } from "pyodide";

On JupyterLite there has been some work on adopting the conda-forge model for making packages for WebAssembly. That work is currently happening in the emscripten-forge organization: https://github.com/emscripten-forge/recipes.
This allows for having more control on which packages should be included, or use different versions than the one provided by Pyodide.
A packed environment (via empack) can then be used in the xeus-python kernel: https://github.com/jupyterlite/xeus-python-kernel

It would be interesting if stlite would allow using a different Python distribution in the web browser, for example via configuration. Or maybe there is already a way to do that via an extension or a plugin?

This could lead to some interesting scenarios where someone could use JupyterLite for authoring a streamlit app, and then run stlite directly in the browser while using the emscripten-forge environment.

Not sure how feasible this would be at the moment but opening the issue to think about it out loud here.

Thanks!

[whitphx]

@jtpio

Thank you for the great suggestion!

it looks like the Web Worker current relies on Pyodide

Yes, currently stlite is relies on Pyodide and it's hardcoded, and as you said, making stlite compatible with other Python runtimes would be so beneficial.

I will take a closer look at this way.
Would it be like below?

• Build an original Python distribution for WebAssembly based on conda, like xeus-python-kernel/jupyterlite_xeus_python/build.py does.
• Replace the current Pyodide with it.

Can you please teach me if you have some suggestions on materials or code to see first to get into it? Thank you!

[jtpio]

Thanks @whitphx.

Right that would likely be the approach. I haven't looked more in details for how it would work in practice with stlite but this would likely require a build step when creating a Streamlit app, to include all the necessary dependencies in the web environment.

As a Jupyter kernel xeus-python adds a communication layer to communicate with the frontend. But it is likely not required for stlite.

There is also this more minimal sample repo that provides a Python REPL in the browser: https://github.com/emscripten-forge/sample-python-repl.
It is based on the emscripten forge packages, pyjs for the Python / JavaScript bindings, and empack to pack a conda environment for the browser.

Some of these tools are still going to evolve quite significantly in the near future. There is also some overlap with what Pyodide provides so there will likely be some convergence on some of the lower levels parts. But it could still be interesting to experiment with it.