-
-
Notifications
You must be signed in to change notification settings - Fork 681
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[pip package] Make loading aware that os.environ can change #198
Comments
yeah because you need to do os.environ['RWKV_CUDA_ON'] = '1' before import rwkv |
Yes, exactly! This is exactly is the issue. Normal imports look like this: (BEGINNING OF FILE)
from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS which is exactly the case with transformers, exllama, llama.cpp, HQQ(which just like RWKV has several backends but only one can be used at the time) Not like this import os
os.environ["RWKV_CUDA_ON"] = f"1"
if "I don't want standard formatter to break my code":
from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS Running autopep8 should not break the code. Which it does for no reasonable reason unless something like this At the very least it's possible to make couple of QoL files like import os
os.environ["RWKV_CUDA_ON"] = "1"
os.environ["RWKV_JIT_ON"] = "1"
if "imports come after os":
from rwkv.model import RWKV
from rwkv.utils import PIPELINE, PIPELINE_ARGS so user code can call Autopep8 is a friend! It shouldn't be worked around |
Expected behavior:
Running something like
autopep8 mycode.py
on the code) that uses RWKV should not break the code.Pressing "ctrl-s" in vs code with "autoformat on save" should not break the code.
It does: linter moves imports where they belong, to the very top of the file, above any statements like
os.environ[]=1
.Actual behavior:
from torch.utils.cpp_extension import load is not executed if environ is not set up during import.
Therefore simple
is unable to load the model
The text was updated successfully, but these errors were encountered: