Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random error when starting docker container again #468

Open
CIB opened this issue Aug 18, 2023 · 0 comments
Open

Random error when starting docker container again #468

CIB opened this issue Aug 18, 2023 · 0 comments

Comments

@CIB
Copy link

CIB commented Aug 18, 2023

When starting petals in docker, sometimes it will fail with the below error.

Steps:

  1. Start petals with the below command
  2. Exit petals with Ctrl-C
  3. Restart the computer
  4. Start petals with the below command again

Immediately re-running the command will succeed without error.

docker run --rm -p 31330:31330 --ipc host --gpus all --volume petals-cache:/cache --rm learningathome/petals:main     python -m petals.cli.run_server --port 31330 stabilityai/StableBeluga2

Error message:

Aug 18 10:18:55.866 [INFO] Running Petals 2.0.0.post3
Aug 18 10:18:56.069 [INFO] A newer version 2.0.1.post2 is available. Please upgrade with: pip install --upgrade petals
Downloading (…)lve/main/config.json: 100%|██████████| 630/630 [00:00<00:00, 6.23MB/s]
Aug 18 10:18:56.764 [INFO] Make sure you follow the LLaMA's terms of use: https://bit.ly/llama2-license for LLaMA 2, https://bit.ly/llama-license for LLaMA 1
Aug 18 10:18:56.765 [INFO] Using DHT prefix: StableBeluga2-hf
Aug 18 10:19:11.835 [INFO] This server is accessible directly
Traceback (most recent call last):
  File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/petals/src/petals/cli/run_server.py", line 224, in <module>
    main()
  File "/home/petals/src/petals/cli/run_server.py", line 208, in main
    server = Server(
  File "/home/petals/src/petals/server/server.py", line 135, in __init__
    self.dht = DHT(
  File "/opt/conda/lib/python3.10/site-packages/hivemind/dht/dht.py", line 88, in __init__
    self.run_in_background(await_ready=await_ready)
  File "/opt/conda/lib/python3.10/site-packages/hivemind/dht/dht.py", line 148, in run_in_background
    self.wait_until_ready(timeout)
  File "/opt/conda/lib/python3.10/site-packages/hivemind/dht/dht.py", line 151, in wait_until_ready
    self._ready.result(timeout=timeout)
  File "/opt/conda/lib/python3.10/site-packages/hivemind/utils/mpfuture.py", line 262, in result
    return super().result(timeout)
  File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/opt/conda/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
FileNotFoundError: [Errno 2] No such file or directory
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant