-
-
Notifications
You must be signed in to change notification settings - Fork 923
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate to "pure Python" Cython mode where applicable #2189
Comments
It's also possible to import a compiled pure python cython module without using the compiled code, by doing a bit of shenanigans with the import machinery. This is useful for tests and benchmarks: _M = TypeVar("_M", bound=ModuleType)
def load_uncompiled_module(module: _M) -> _M:
"""Load the non-compied version of a module that is also
compiled with cython.
"""
full_name = module.__name__
assert module.__spec__
parent_name = module.__spec__.parent
assert parent_name
parent_module = sys.modules[parent_name]
assert parent_module.__spec__
package_path = parent_module.__spec__.origin
assert package_path and package_path.endswith("__init__.py")
name = full_name.split(".")[-1]
module_path = package_path.replace("__init__.py", f"{name}.py")
py_spec = importlib.util.spec_from_file_location(full_name, module_path)
assert py_spec
py_module = importlib.util.module_from_spec(py_spec)
assert py_spec.loader
py_spec.loader.exec_module(py_module)
return cast(_M, py_module) |
One unpleasant gotcha so far is that |
Another glitch was that some aliases were missing, but this will be fixed in the next Cython release: cython/cython#5934. |
As of Cython 3.0+, it is possible to write most of the Cython code as pure Python, leveraging decorators and type annotations to convey the desired C compilation behaviour.
This would require
cython
at runtime, but we can shim only the annotations in order to avoid a hard dependency. @CaselIT has already implemented a prototype in this vein for SQLAlchemy.We might want to keep
.pyx
for some tight-knit C character loops used for parsing parameters, but, for instance, multipart stream reader is mostly about typing withPy_ssize_t
, and calling internal helpers in C, so we should be able to easily rewrite it in such mixed mode.The main benefit is obviously that we don't need to maintain two different codebases for the same thing. Also, even if we keep dual implementations, we would be able to easily test implementation in pure Python unit tests, tracking coverage.
The text was updated successfully, but these errors were encountered: