Add docker support #1239
Add docker support #1239
Conversation
|
thanks for picking this up!
don't modify do we need default |
|
@vladmandic I have now removed Regarding |
|
just an idea - having data inside container is really against the concept of containers. for example: |
|
Good idea, I have made it mandatory now |
|
looks good to me, but please tell me you've actually tested it? :) |
|
I built it again from scratch and noticed an error ^^ |
|
Can you guys talk about the security benefits/pros and cons of using this? |
talk about benefits of using docker in general? not really, that's really outside of the scope of this pr, this is to provide simple-to-use template. |
|
@vladmandic I think its ready to be merged |
|
I merged
|
|
I would also suggest making the first argument to the entrypoint |
| environment: | ||
| DATA_DIR: "./data" | ||
| volumes: | ||
| - ./data:/webui/data |
There was a problem hiding this comment.
Solves the reinstall problem noted here.
| - ./data:/webui/data | |
| - ./data:/webui/data | |
| - ./venv/lib:/webui/venv/lib | |
| - ./repositories:/webui/repositories |
There was a problem hiding this comment.
Even with the above, i still see the following on re-create but i'm not sure where they are coming from:
Downloading (…)olve/main/vocab.json: 100% 961k/961k [00:00<00:00, 34.7MB/s]
Downloading (…)olve/main/merges.txt: 100% 525k/525k [00:00<00:00, 43.1MB/s]
Downloading (…)cial_tokens_map.json: 100% 389/389 [00:00<00:00, 2.82MB/s]
Downloading (…)okenizer_config.json: 100% 905/905 [00:00<00:00, 3.26MB/s]
Downloading (…)lve/main/config.json: 100% 4.52k/4.52k [00:00<00:00, 13.2MB/s]
There was a problem hiding this comment.
It would be more clean to declare those folders as VOLUMEs in the Dockerfile. Then you could even leave out the bind mounts in the compose file so they will be created as anonymous volumes during launch.
There was a problem hiding this comment.
/webui/venv/lib contains files that were installed when building the image, so I think it would require additional changes to mount it
Co-authored-by: Jakub Sztandera <[email protected]>
|
Very unfortunate that the |
|
@Kubuxu thanks for the suggestions, I've fixed the env vars.
Could you clarify what you meant by this? Essentially running |
Correction, not But in essence, having the default run command in So for example ENTRYPOINT ["/bin/bash", "-c", "${INSTALLDIR}/entrypoint.sh \"$0\" \"$@\""] # same as today
CMD ["webui"] Then the #!/usr/bin/env bash
set -e
if [ "$1" = 'postgres' ]; then
chown -R postgres "$PGDATA"
if [ -z "$(ls -A "$PGDATA")" ]; then
gosu postgres initdb
fi
shift
exec gosu postgres "$@"
fi
exec "$@"This will allow the user to both pass params to the launch.py like this: |
Co-authored-by: Jakub Sztandera <[email protected]>
|
@Kubuxu I think what you want to do here is already possible using the |
|
Yeah, this is another way of doing this. We can go down the |
Dockerfile
Outdated
| @@ -0,0 +1,51 @@ | |||
| FROM nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu22.04 | |||
There was a problem hiding this comment.
I propose allowing to use different CUDA versions by using the following instead (adapted from llama.cpp):
ARG UBUNTU_VERSION=22.04
# This needs to generally match the container host's environment.
ARG CUDA_VERSION=11.8.0
# Target the CUDA runtime image
ARG BASE_CUDA_CONTAINER=nvidia/cuda:${CUDA_VERSION}-cudnn8-runtime-ubuntu${UBUNTU_VERSION}
FROM ${BASE_CUDA_CONTAINER}There was a problem hiding this comment.
I have implemented this now, although the user needs to be careful to specify compatible cuda and ubuntu versions.
| fi | ||
|
|
||
| # Ensure that potentially bind-mounted directories are owned by the user that runs the service | ||
| chown -R $RUN_UID:$RUN_UID $DATA_DIR |
There was a problem hiding this comment.
I propose to add these lines as well. Otherwise image generation works only if the option Always save all generated images is enabled.
# Create directory for temporary files and assign it to the user that runs the service
mkdir /tmp/gradio
chown -R $RUN_UID:$RUN_UID /tmp/gradio
|
|
||
| # Install automatic1111 dependencies (installer.py) | ||
| RUN . $INSTALLDIR/venv/bin/activate && \ | ||
| python installer.py && \ |
There was a problem hiding this comment.
I don't think this does anything, installer.py looks like it doesn't run any code if one tries to run it directly.
I think python launch.py --test is what you would want, except it installs the CPU-only version of Torch because docker compose build doesn't support runtimes if I understood it correctly. There is a --use-cuda flag but it doesn't actually force the use of cuda, but perhaps installer.py could be modified so that it does?
There was a problem hiding this comment.
there are plenty of flags that can be used as-is, there is even --skip-torch. but installing packages AND skipping torch is not viable since plenty of packages down the list require torch, so they would pull it in.
There was a problem hiding this comment.
I think the goal here is to install torch and other dependencies while building the docker image, so skipping it wouldn't be of much use :) Bypassing the nvidia-smi check when --use-cuda is present makes this work for me at least.
The arg description says "force use nVidia CUDA backend", so IMO it would be ok to skip the check and crash if it doesn't work, but it's of course up to you to decide how you want your app to behave.
There was a problem hiding this comment.
Don't assume docket image is for Nvidia only, but it's OK to require one of --use-xxxparams to be provided.
|
I had issues building this from within Ubuntu (20.04). I'm going to document my experience so that you can see the troubles I had along the way to hopefully help me fix them, but ultimately fix it for others who might use it once this has been accepted as a merge. Please don't take this as negative criticism at all, cos I really do appreciate all the hard work you guys are putting into this! I hope my experiences can help to get this accepted. I just wish I knew more to help move things along. I kept getting the error: which I fixed by changing the You can also confirm it using the command After I got past that error by removing the name variable, this was the error I got: For some reason And I changed it to: Although I changed it to 20.04 and 12.1.0 (which I confirmed by going to: https://hub.docker.com/r/nvidia/cuda/tags?page=1&name=12.1.0-cudnn8-runtime-ubuntu), I'm pretty sure changing it to: Would work fine since that does exist too: https://hub.docker.com/r/nvidia/cuda/tags?page=1&name=11.8.0-cudnn8-runtime-ubuntu The main issue seems to be with The next issue is the tzdata, it would be good to set a default during installation with an But when you type in 8 and hit enter, nothing happens. I had to stop the instance in portainer, and recreate it but with the But once those were in, and it finished setting up. It just stopped running. Trying to re-run it, it obviously continues where it left off because all the packages are installed and tzdata is already set up, and then stops straight away. Trying to diagnose what the last message was and docker says there are no logs it can access for it. Re-running it in the terminal again to make sure I didn't miss anything and I get: Oh! So must be the GPU permission, but still: Slightly more information, tried it with the Hmmm... tried the nvidia test using the same base cuda I used for the installation of So everything is set up fine for docker, but the image still isn't working. Not sure where I am going wrong, but I feel like I'm close! Note that I pulled this from the master branch on Edit: I realised after submitting that I hadn't tried the proper image for the nvidia test of |
|
Didn't want to give up, so tried one more time - scrapped and purged everything, reset the repo back to how it was, and did This time I got a lot further! It installed correctly, run through everything. But this time in the terminal it looked like it had stopped doing anything after Started up another terminal and attached to the running image, and saw a different message saying Then afterwards I got: Followed by a long traceback log, but it looked like it was still going and did... And that's when it exited. Re-running So, still not working, but at least it was a derp moment on my part for putting in lower ubuntu version and a higher cuda version. There does appear to be an issue getting some of the needed dependencies, such as the extensions (although not fully required technically to get it working), and loading up the |
|
I think docker-compose has been deprecated in favour of "docker compose". IIRC that ought to solve the top-level name tag error. |
|
@JohanAR Sure, you're not wrong that "docker compose" is the preferred method and "docker-compose" is deprecated and is a stub for legacy reasons to "docker compose" in the latest versions of docker. However, NVIDIA CUDA Toolkit is only supported on Docker 20.10.x (ref: nvidia install guide. Which meant I had to downgrade to 20.10 a long while back to get anything CUDA working without some hacky workaround. And the Which means, most people should be running on docker 20.10.x if they want to have the CUDA toolkit on Linux properly, or even in the cloud for that matter. And I believe those on Windows will likely experience similar issues since that recommends going through the WDL2 route. There are workarounds to this obviously on the latest version of docker, which as far as I understand crashes on the latest-latest (which means you have to always be running a slightly older version of 23.x.x or 24.x.x) but that would mean this repo would need to support said workarounds or other people will post issue after issue that it isn't working for them. I'm going through the process of upgrading back to the latest version - cos I would love to be proved wrong - and will report back my findings, but I suspect I will end up having to figure a bunch of workarounds to get it to work properly. |
I used it with Docker 23 as well as now 24, with Ubuntu 22.04 and now 23.04, using the apt package source It worked flawlessly out-of-the-box and I did not experience problems. IMHO there is no reason to keep using an old Docker version. |
|
Running in to the same issue as @hazrpg, it fails when "Initializing middleware". I'm not sure what the Python code is doing, but it seems to be missing some configuration attributes, maybe? Configuration Also, is it possible to pass a flag to avoid the prompt "Download the default model? (y/N)" ? The reason I'm asking is that it's quite uncommon to have to attach to the running container to answer setup parameters. It works but it's not usual with Docker builds. |
|
|
Firstly, thanks to everyone for the great work put into vladmandic/automatic! I'm recording my experiences trying to use the Dockerfile with vast.ai in case it is useful for others. My apologies if the approach I took was not best practices or just plain wrong - I'm fairly new to docker so please take the following as the experiences of a naive end-user trying to get this to work on a GPU cloud provider. My use case is that I have a Macbook Pro but I would like to build and use a docker image of vladmandic/automatic that can be used on a GPU cloud provider like vast.ai or runpod.io. My config: Steps:
Results:
I'm happy to continue testing on vast.ai if someone can provide a linux image or instructions for how to successfully build a linux image from this repo on a MBP. |
I did eventually try the upstream docker apt packages, instead of the Canonical/Debian ones. Looks like although the Nvidia toolkit says it doesn't support newer versions, the lovely docker peeps must have gotten around that and made sure it still works. So I stand corrected, thank you for pointing it out. However I'm still stuck at the middleware stage sadly even with the newer docker and using |
|
Why did the PR stall? Was there a technical difficulty? |
|
moving status to draft until comments are incorporated and maintainer is found. |
|
What is the status of this PR? Using SD.next with docker install would be a huge win IMHO. |
|
there are plenty of users using sdnext inside a docker container, but having an official dockerfile is tricky as everyone has their own idea what docker config should be like and it also varies on platform. |
|
On that note for anyone looking for a "one-click" docker deploy -- I have contributed to and am using grokuku/stable-diffusion on a linux host with nvidia gpu. It "just works" and stays up-to-date with master branch automatically. Read the readme ofc but an example run command: |
Thanks for the link, will try soon. |
|
I currently have a WIP branch right here for an NVIDIA CUDA-based docker image. Also installs onediff, tensorflow, and working on adding TensorRT support. It boots and works perfectly, but still tweaking it. Any contributions welcome for anyone who wants to add AMD/Intel/whatever support, I only have NVIDIA hardware so I can't test on other platforms. Broadly, the |
Description
Add support to run the UI using docker.
Since the previous PRs (#403, #844) stalled, I merged their approaches and fixed remaining issues.
Notes
To improve security, the process is run using a non-root user inside the container. Since bind-mounts are owned by
rootinside the container, theentrypoint.shscript changes ownership to the non-root user to make it writable.Environment and Testing