Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When attempting to add the CohereForAI/aya-101 model using CohereForAI's Aya model, an error occurred during the loading process. #910

Open
yasserkh2 opened this issue Feb 20, 2024 · 0 comments
Open

When attempting to add the CohereForAI/aya-101 model using CohereForAI's Aya model, an error occurred during the loading process. #910

yasserkh2 opened this issue Feb 20, 2024 · 0 comments

Comments

@yasserkh2
Copy link

Describe the bug

When attempting to add the CohereForAI/aya-101 model using CohereForAI's Aya model, an error occurred during the loading process.
bentoml.exceptions.BentoMLException: Failed to load bento model because it does not contain a 'model.yaml'

To reproduce

1-add the model like they describe here [https://github.com/bentoml/OpenLLM/blob/main/ADDING_NEW_MODEL.md]
2-pip install openllm
3-openllm -h
4-Run the command to add the TRUST_REMOTE_CODE=True openllm start CohereForAI/aya-101 model.

Logs

It is recommended to specify the backend explicitly. Cascading backend might lead to unexpected behaviour.
Traceback (most recent call last):
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 647, in open
    return io.open(
FileNotFoundError: [Errno 2] No such file or directory: b'/home/ubuntu/bentoml/models/vllm-cohereforai--aya-101/latest'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 144, in get
    _tag.version = self._fs.readtext(_tag.latest_path())
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/base.py", line 693, in readtext
    self.open(
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 643, in open
    with convert_os_errors("open", path):
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/error_tools.py", line 89, in __exit__
    reraise(fserror, fserror(self._path, exc=exc_value), traceback)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/six.py", line 718, in reraise
    raise value.with_traceback(tb)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 647, in open
    return io.open(
fs.errors.ResourceNotFound: resource 'vllm-cohereforai--aya-101/latest' not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 647, in open
    return io.open(
FileNotFoundError: [Errno 2] No such file or directory: b'/home/ubuntu/bentoml/models/vllm-cohereforai--aya-101/7185b1d43433ec1b0a0039b043ea194ef6bc0a06/model.yaml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/models/model.py", line 221, in from_fs
    with item_fs.open(MODEL_YAML_FILENAME, "r") as model_yaml:
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/wrapfs.py", line 472, in open
    open_file = _fs.open(
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 643, in open
    with convert_os_errors("open", path):
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/error_tools.py", line 89, in __exit__
    reraise(fserror, fserror(self._path, exc=exc_value), traceback)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/six.py", line 718, in reraise
    raise value.with_traceback(tb)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/fs/osfs.py", line 647, in open
    return io.open(
fs.errors.ResourceNotFound: resource 'model.yaml' not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/python/yasser/n_venv/bin/openllm", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/openllm_cli/entrypoint.py", line 160, in wrapper
    return_value = func(*args, **attrs)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/openllm_cli/entrypoint.py", line 141, in wrapper
    return f(*args, **attrs)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/openllm_cli/entrypoint.py", line 366, in start_command
    llm = openllm.LLM(
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/openllm/_llm.py", line 223, in __init__
    model = bentoml.models.get(self.tag)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/simple_di/__init__.py", line 139, in _
    return func(*_inject_args(bind.args), **_inject_kwargs(bind.kwargs))
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/models.py", line 45, in get
    return _model_store.get(tag)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 149, in get
    self._recreate_latest(_tag)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 118, in _recreate_latest
    items = self.list(tag.name)
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 106, in list
    return [self._get_item(t) for t in tags]
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 106, in <listcomp>
    return [self._get_item(t) for t in tags]
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/store.py", line 114, in _get_item
    return self._item_type.from_fs(self._fs.opendir(tag.path()))
  File "/home/ubuntu/python/yasser/n_venv/lib/python3.10/site-packages/bentoml/_internal/models/model.py", line 224, in from_fs
    raise BentoMLException(
bentoml.exceptions.BentoMLException: Failed to load bento model because it does not contain a 'model.yaml'

Environment


#### System information

`bentoml`: 1.1.11
`python`: 3.10.9
`platform`: Linux-5.15.0-1052-aws-x86_64-with-glibc2.31
`uid_gid`: 1000:1000
<details><summary><code>pip_packages</code></summary>

<br>

accelerate==0.27.2
aiohttp==3.9.3
aioprometheus==23.12.0
aiosignal==1.3.1
annotated-types==0.6.0
anyio==4.3.0
appdirs==1.4.4
asgiref==3.7.2
asttokens==2.4.1
async-timeout==4.0.3
attrs==23.2.0
bentoml==1.1.11
bitsandbytes==0.41.3.post2
build==0.10.0
cattrs==23.1.2
certifi==2024.2.2
charset-normalizer==3.3.2
circus==0.18.0
click==8.1.7
click-option-group==0.5.6
cloudpickle==3.0.0
coloredlogs==15.0.1
comm==0.2.1
contextlib2==21.6.0
cuda-python==12.3.0
cupy-cuda12x==12.1.0
datasets==2.17.1
debugpy==1.8.1
decorator==5.1.1
deepmerge==1.1.1
Deprecated==1.2.14
dill==0.3.8
distlib==0.3.8
distro==1.9.0
einops==0.7.0
exceptiongroup==1.2.0
executing==2.0.1
fastapi==0.109.2
fastcore==1.5.29
fastrlock==0.8.2
filelock==3.13.1
filetype==1.2.0
frozenlist==1.4.1
fs==2.4.16
fsspec==2023.10.0
ghapi==1.0.4
h11==0.14.0
httpcore==1.0.3
httptools==0.6.1
httpx==0.26.0
huggingface-hub==0.20.3
humanfriendly==10.0
idna==3.6
importlib-metadata==6.11.0
inflection==0.5.1
ipykernel==6.29.2
ipython==8.21.0
jedi==0.19.1
Jinja2==3.1.3
jsonschema==4.21.1
jsonschema-specifications==2023.12.1
jupyter_client==8.6.0
jupyter_core==5.7.1
markdown-it-py==3.0.0
MarkupSafe==2.1.5
matplotlib-inline==0.1.6
mdurl==0.1.2
mpmath==1.3.0
msgpack==1.0.7
multidict==6.0.5
multiprocess==0.70.16
mypy-extensions==1.0.0
nest-asyncio==1.6.0
networkx==3.2.1
ninja==1.11.1.1
numpy==1.26.4
nvidia-cublas-cu12==12.1.3.1
nvidia-cuda-cupti-cu12==12.1.105
nvidia-cuda-nvrtc-cu12==12.1.105
nvidia-cuda-runtime-cu12==12.1.105
nvidia-cudnn-cu12==8.9.2.26
nvidia-cufft-cu12==11.0.2.54
nvidia-curand-cu12==10.3.2.106
nvidia-cusolver-cu12==11.4.5.107
nvidia-cusparse-cu12==12.1.0.106
nvidia-ml-py==11.525.150
nvidia-nccl-cu12==2.18.1
nvidia-nvjitlink-cu12==12.3.101
nvidia-nvtx-cu12==12.1.105
openllm==0.4.44
openllm-client==0.4.44
openllm-core==0.4.44
opentelemetry-api==1.20.0
opentelemetry-instrumentation==0.41b0
opentelemetry-instrumentation-aiohttp-client==0.41b0
opentelemetry-instrumentation-asgi==0.41b0
opentelemetry-sdk==1.20.0
opentelemetry-semantic-conventions==0.41b0
opentelemetry-util-http==0.41b0
optimum==1.17.1
orjson==3.9.14
packaging==23.2
pandas==2.2.0
parso==0.8.3
pathspec==0.12.1
pexpect==4.9.0
pillow==10.2.0
pip-requirements-parser==32.0.1
pip-tools==7.3.0
platformdirs==4.2.0
prometheus_client==0.20.0
prompt-toolkit==3.0.43
protobuf==4.25.3
psutil==5.9.8
ptyprocess==0.7.0
pure-eval==0.2.2
pyarrow==15.0.0
pyarrow-hotfix==0.6
pydantic==2.6.1
pydantic_core==2.16.2
Pygments==2.17.2
pynvml==11.5.0
pyparsing==3.1.1
pyproject_hooks==1.0.0
python-dateutil==2.8.2
python-dotenv==1.0.1
python-json-logger==2.0.7
python-multipart==0.0.9
pytz==2024.1
PyYAML==6.0.1
pyzmq==25.1.2
quantile-python==1.1
ray==2.9.2
referencing==0.33.0
regex==2023.12.25
requests==2.31.0
rich==13.7.0
rpds-py==0.18.0
safetensors==0.4.2
schema==0.7.5
scipy==1.12.0
sentencepiece==0.2.0
simple-di==0.1.5
six==1.16.0
sniffio==1.3.0
stack-data==0.6.3
starlette==0.36.3
sympy==1.12
tokenizers==0.15.2
tomli==2.0.1
torch==2.1.2
tornado==6.4
tqdm==4.66.2
traitlets==5.14.1
transformers==4.37.2
triton==2.1.0
typing_extensions==4.9.0
tzdata==2024.1
urllib3==2.2.1
uvicorn==0.27.1
uvloop==0.19.0
virtualenv==20.25.0
vllm==0.3.1
watchfiles==0.21.0
wcwidth==0.2.13
websockets==12.0
wrapt==1.16.0
xformers==0.0.23.post1
xxhash==3.4.1
yarl==1.9.4
zipp==3.17.0


</details>
_______________________________________________________________________________________________________

- `transformers` version: 4.37.2
- Platform: Linux-5.15.0-1052-aws-x86_64-with-glibc2.31
- Python version: 3.10.9
- Huggingface_hub version: 0.20.3
- Safetensors version: 0.4.2
- Accelerate version: 0.27.2
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed


### System information (Optional)

_No response_
@yasserkh2 yasserkh2 changed the title bug: When attempting to add the CohereForAI/aya-101 model using CohereForAI's Aya model, an error occurred during the loading process. Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@yasserkh2