Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

asyncio functionality is mostly unusable after async handling was applied #93

Open
pmeier opened this issue Feb 2, 2024 · 2 comments · May be fixed by #94
Open

asyncio functionality is mostly unusable after async handling was applied #93

pmeier opened this issue Feb 2, 2024 · 2 comments · May be fixed by #94

Comments

@pmeier
Copy link
Contributor

pmeier commented Feb 2, 2024

Reporting on my own failures here. In #90 we added automatic async handling and did so by using asyncio.run:

import asyncio as __asyncio__
__async_wrapper_locals__ = __asyncio__.run(__async_wrapper__())

The problem is, and I didn't know this before, that asyncio.run is meant to be only called once. This is what the documentation says about this:

This function should be used as a main entry point for asyncio programs, and should ideally only be called once.

However, we call it for every code block that needs async handling. That in itself is not an issue as the gallery here builds just fine. Unfortunately, it breaks down when other libraries that are part of the documentation building also need to use some async functionality.

For example, panel creates an asyncio.Lock on import. When building some async galleries before the import, we effectively have the following situation

import asyncio
import asyncio.events

async def foo():
    pass

asyncio.run(foo())

loop = asyncio.events.get_event_loop()

And the get_event_loop fails with

RuntimeError: There is no current event loop in thread 'MainThread'.

because we have previously used asyncio.run.

My absolute crude workaround for now is the following snippet that has to be placed before any further asyncio functionality is used:

import asyncio

asyncio.get_event_loop_policy()._local._set_called = False

Please note that this works for my case, but I have no idea if this screws up other cases as we are messing with low-level private attributes here.

I'll look on how to restructure the async handling to avoid using asyncio.run to avoid this mess all together.

@pmeier
Copy link
Contributor Author

pmeier commented Feb 2, 2024

A simple solution is to replace

__async_wrapper_locals__ = __asyncio__.run(__async_wrapper__())

with

__loop__ = __asyncio__.new_event_loop()
try:
    __async_wrapper_locals__ = __loop__.run_until_complete(__async_wrapper__())
finally:
    __loop__.close()

For our purpose this is equivalent to asyncio.run, but doesn't set the _set_called flag that prevents more event loops from being created.

We could also have one global event loop instead of recreating it every time for the async handling. I'll have a look at how difficult it would be to fit in.

@pmeier
Copy link
Contributor Author

pmeier commented Feb 2, 2024

Here is what IPython does. They have a function that resembles asyncio.get_event_loop (deprecated in 3.12) to potentially create and return a singleton global event loop. This event loop is then used to just call run_until_complete on it.

We could potentially vendor this function and always insert it into the globals of the galleries. That way we can just call

__get_asyncio_loop__().run_until_complete(__async_wrapper__())

in our wrapper.

@pmeier pmeier linked a pull request Feb 2, 2024 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant