approach to binding a directory (from host machine) to containers $HOME directory

[capsulecorplab]

Opening up this discussion thread for users who have or may want to build and/or run images configured via docker-compose and share use cases with example docker-compose.yml files

Updated by @accetto:
I've renamed this thread from Building/running images from docker-compose to approach to binding a directory (from host machine) to containers $HOME directory because the actual discussion has turned in that direction. I'll open a new thread named Building/running images from docker-compose.

[capsulecorplab]

Here's one such use case I'd like to explore #40

[accetto]

Hello @capsulecorplab,

maybe you could rename this thread because the discusion has shifted towards binding the whole $HOME directory.

Then you can open the original subject Building/running images from docker-compose as a new thread.

What do you think?

Regards,
accetto

[accetto]

I've renamed the thread (see above).

[accetto]

Please note that this is a continuation of my answer to the question from the issue #40.

I'll split it into several posts, so we can comment each part separately.

The procedure described here works starting from the release 23.03.2

---

If you don't need to extend the image and you really want to bind the whole $HOME directory, then my recommended yaml file would be something like this:

version: "3"

volumes:
  my-home-volume

services:
  ubuntu-vnc-xfce-g3:
    image: accetto/ubuntu-vnc-xfce-g3:latest
    ports:
      - "25901:5901
      - "26901:6901
    environment:
      VNC_PW: "password"
    volumes:
      - type: volume
        source: my-home-volume
        target: /home/headles

[accetto]

If you'll bind the whole $HOME directory, then you'll probably get the problems that are described in the issues #36 and #37. I've tested it on Windows and Linux, with and without the Docker Desktop and it was always caused by biding the whole $HOME directory.

You can mitigate the black screen problem after the container restart by installing the package libgl1, but your profile will never be correctly initaliazed, if you bind the complete $HOME directory to an external folder on the host.

This is a big difference comparing to the managed environments, as for example Kasm Workspaces.

However, the profile will be initialized correctly, if you'll use Docker volumes, as I do in my recommended yaml above.

If you're willing to do some manual work, then you can use the following little trick for initializing your profile in bound directories. However, I haven't had enough time to test it by longer use.

It would would be really nice if some of you could help with that. We need to test, how newly created files will be handled. There are pretty big differences in handling the permissions in bound directories on different enwironments.

If the pattern will work correctly, then I could automate it later. So see the next post.

[accetto]

Let's use the following helper.yml file:

version: "3"

volumes:
  my-home-volume:

services:

  init:
    profiles:
      - init-home
    image: accetto/ubuntu-vnc-xfce-g3:latest
    ports:
      - "25901:5901"
      - "26901:6901"
    environment:
      VNC_PW: "password"
    volumes:
      - type: volume
        source: my-home-volume
        target: /home/headless
      - type: bind
        source: ~/shared/my-home
        target: /home/headless-copy

  final:
    profiles:
      - shared-home
    image: accetto/ubuntu-vnc-xfce-g3:latest
    ports:
      - "25901:5901"
      - "26901:6901"
    environment:
      VNC_PW: "password"
    volumes:
      - type: bind
        source: ~/shared/my-home/headless
        target: /home/headless

The yaml file contains the configuration of two containers that are used in a sequence, as it's described in the next post.

Warning! Be sure to use the same password for both containers, otherwise the final container will enter the view-only mode.

Be also aware, that on Linux you have to create the folder ~/shared/my-home manually before you create the first container. On Windows, on the other hand, it'll be created automatically. Note also that we assume a fresh new folder in both cases.

[accetto]

Step 1

We'll use the first container to initialize the $HOME directory. Because it happens inside a Docker volume, all files will be initialized correctly.

Let's call our compose project test. Then we can create the first container like this:

docker compose --profile init-home -p test -f helper.yml up -d

A container test-init-1 should be created and it should keep running in the background.

[accetto]

Step 2

Connect to the container by a VNC viewer (using the port 25901) or a web browser (using the port 26901). Note that the VNC password is empty in this case.

You should see the desktop similar to this:

Make sure that you see the "Version Sticker" icon on the desktop. If you don't, then something went wrong.

Then you check the content of the /home/headless directory. It should look like this:

If it doesn't, then something went wrong.

Move one level up to the parent /home directory. It should look like this:

[accetto]

Step 3

Copy the complete /home/headless directory to the bound folder on the host.

Open the terminal and from inside the /home directory execute the following command:

cp -r headless/ headless-copy/

Be sure to make the copy from inside the container. If you do it using the docker cp command, then not all files will be copied correctly.

After copying the $HOME directory we don't need the helper container any more and we can remove it, including the Docker volume:

docker compose --profile init-home -p test -f helper.yml down --volumes

Be sure to remove the helper container before creating the final one, because they share the same ports. Optionally you can also modify the yaml file, of course.

[accetto]

Step 4

Now we can create the final container that will use the shared $HOME directory in the bound external directory on the host.

docker compose --profile shared-home -p test -f helper.yml up -d

Container test-shared-1 should be created and it should keep running in the background.

You can connect to it the same way as it's described above, using the same ports. You should also see the same desktop and the same content of the /home/headless folder.

Note that the VNC password is this time password.

[accetto]

Final remarks

This approach should work on Windows and also on Linux.

I've tested it in following environments:

• Windows 10 with Docker Desktop and WSL2 (Ubuntu 22.04)
• Debian 11 with Docker Desktop
• Debian 11 without Docker Desktop

However, as I've mentioned above, I didn't have enough time for longer testing yet.

Especially adding new files from inside and outside the container is a big concern, because the file permissions are handled really differently in different environments. It's probable, that some permission adjustments will be necessary.

Personally I would not recommend to bind the whole $HOME directory to external folders on local host. I recommend to bind individual subfolders.

Binding to external directories was never without challenges and it will probably stay that way also in the future. The environments are simply too different.

It's a big difference comparing to the managed environments, as for example Kasm Workspaces.

Any help and feedback are welcome.

[markNZed]

I would like to bind the home directory because it makes it easy to encrypt at rest on a medium that is not part of the Docker install. Can you provide more details about why a home directory cannot use bind ?

I did have this working a few weeks ago (Ubuntu WSL2 and Windows 10) and am now running into an issue where the home directory appears to be empty inside the container i.e. the bind is not working but no error messages.

A minor point: should there be a "headless" group with ID 1000 created?

Later... I seem to have the mount working again but it is hard to know why. I suspect it might be the directory I am mounting to the docker container is itself mounted via WSL and I ran explorer.exe . from the directory in WSL. Maybe Windows 10 needs to "see" that directory before Docker Desktop can use it.

[accetto]

Binding Docker containers to external folders is a tricky area, believe me. :)

The behaviour you've encountered could have been caused by the fact, that you've bound a non-empty folder. The original folder in the container was hidden then. But it could be also something else. I never tested that particular configuration.

I'm actually not quite sure where your bound folder is. Is it created in the Windows file system or in the WSL's file system?

Btw, I usually use terminal for managing the files inside the WSL2 file system. You can also use Thunar if you've installed WSLg.

[accetto]

Can you provide more details about why a home directory cannot use bind ?

I'm not saying that you cannot bind the home directory technically. I'm only saying that it's tricky with Docker containers. It's also inconsistent and unreliable across the environments. It's also often changing with Docker Desktop updates.

You should not compare it with managed environments like, for example, Kasm Workspaces.

If you want it working, you have to manage your environment yourself. And you have to be aware, that every update can change the landscape without considering your wishes. :)

The biggest problem is the intrinsic incompatibility between the Windows and Linux file systems, meaning the file permissions. If you'll experiment further, you'll also find, that even on Linux it makes a difference, if you use the Docker Desktop or not.

There is a big difference between using and initializing a shared directory tree, especially the $HOME one.

When the container is created, a lot of files and folders get also created. Some of them get special permissions and some even special content, usually randomly generated. These new files and folders modify/extend the container's file system, which has been originally built-in into the image. There are also usually at least two different users inside the container involved (root and the container user), that are somehow mapped to the users on the host (depends on the environment).

Then the bound folder on the host can be an existing or not-existing-yet one. It can be empty or not empty. It can have default permissions for the particular environment or modified ones. It can belong to the host user running the container or not, etc. etc.

The file systems differ, the environments differ and the Docker handling also differs. The pages Volumes and Bind mounts describe the differences. Check particulary the rules for initializing of empty/non-empty folders.

For this particular discussion I've tested many scenarios on both Windows and Linux and the only working option across the environments was using the Docker volumes. This matches what Docker itself recommends, even if I would like to have more choices. :)

The requirement you've described is a good reason for using bind mounts.

One thing you can do, is to try to set the permissions manually, which is however environment and often also application dependent.

I would really recommend to use the Docker volumes or to bind the individual working folders of the individual applications. I also use this kind of combination.

For example, I usually bind my Firefox profiles to an external folder, because it works pretty well if you create the parent folder manually and give it "loose enough" permissions. The Firefox then creates the content on its first start and sets the correct permissions itself.

On the other hand, after I've updated my GitLab container the last time, the previous shared folders didn't work any more. I had to use a combination of Docker volumes and bind mounts.

The procedure I've described above could be the first step in developing some helper tools, maybe. I would be glad, if you could try it. Maybe it'll cover your requirements.

[accetto]

A minor point: should there be a "headless" group with ID 1000 created?

I'm not sure what you mean. Are you asking if there is such a group or why is it there?

Btw, you can check the basic stuff by running the following test from the $HOME directory:

./tests/test-01.sh

You should get the following output:

+ id
uid=1000(headless) gid=1000(headless) groups=1000(headless)
+ ls -l /etc/passwd /etc/group
-rw-r--r-- 1 root root  481 Mar 21 17:35 /etc/group
-rw-r--r-- 1 root root 1029 Mar  6 18:11 /etc/passwd
+ tail -n2 /etc/passwd
messagebus:x:101:101::/nonexistent:/usr/sbin/nologin
headless:x:1000:1000:Default:/home/headless:/bin/bash
+ tail -n2 /etc/group
messagebus:x:101:
headless:x:1000:
+ ls -ld /dockerstartup /home /home/headless
drwxr-xr-x 1 headless headless 4096 Mar 21 17:35 /dockerstartup
drwxr-xr-x 1 root     root     4096 Mar  6 18:02 /home
drwxr-xr-x 1 headless headless 4096 Mar 24 18:42 /home/headless
+ ls -l /dockerstartup
total 60
-rw-r--r-- 1 headless headless 3090 Feb 20 16:21 help.rc
-rw-r--r-- 1 headless headless  467 Mar 24 10:10 novnc.log
-rw-r--r-- 1 headless headless 6721 Feb 20 16:21 parser.rc
-rwxr--r-- 1 headless headless  872 Feb 20 16:21 set_user_permissions.sh
-rwxr-xr-x 1 headless headless 4700 Feb 20 16:21 startup.sh
-rw-r--r-- 1 headless headless 3877 Feb 20 16:21 user_generator.rc
-rwxr--r-- 1 headless headless 5307 Feb 22 14:39 version_of.sh
-rwxr--r-- 1 headless headless 3362 Feb 20 16:21 version_sticker.sh
-rw-r--r-- 1 headless headless 5515 Mar 24 13:58 vnc.log
-rw-r--r-- 1 headless headless 4958 Feb 20 16:21 vnc_startup.rc
+ mkdir -p /home/headless/new-dir
+ touch /home/headless/new-file
+ ls -l /home/headless
total 64
drwxrwxrwx 1 headless headless 4096 Mar 24 10:34 Bookmarks
drwxr-xr-x 1 headless headless 4096 Mar  6 18:11 Desktop
drwxr-xr-x 2 headless headless 4096 Mar 21 17:35 Documents
drwxrwxrwx 1 headless headless 4096 Mar 11 11:25 Downloads
drwxr-xr-x 2 headless headless 4096 Mar 21 17:35 Music
drwxr-xr-x 2 headless headless 4096 Mar 21 17:35 Pictures
drwxr-xr-x 2 headless headless 4096 Mar 21 17:35 Public
drwxr-xr-x 2 headless headless 4096 Mar 21 17:35 Templates
drwxr-xr-x 2 headless headless 4096 Mar 21 17:35 Videos
drwxr-xr-x 1 headless headless 4096 Mar 21 17:30 firefox.plus
drwxr-xr-x 2 headless headless 4096 Mar 24 18:42 new-dir
-rw-r--r-- 1 headless headless    0 Mar 24 18:42 new-file
-rw-r--r-- 1 headless headless  458 Feb 20 16:21 readme-firefox-plus.md
-rw-r--r-- 1 headless headless  185 Feb 20 16:21 readme.md
-rw-r--r-- 1 headless headless 1364 Mar 24 18:42 test-01.log
drwxr-xr-x 1 headless headless 4096 Mar  6 18:11 tests

[markNZed]

The bound folder is in WSL's file system (it is a LUKS encrypted SSD mounted in WSL). The reason I ran explorer.exe was a bit random - I was just trying to see if the Docker Desktop might somehow work better if I tried to mount to the Windows path but I never tried that as it started working. If you launch 'explorer.exe .' in a WSL terminal then it will launch Windows explorer in that directory via the Windows path to the WSL directory.

I don't think there should be a file systme incompatibility because I'm running in WSL2 so everything is in Linux. I agree that trying to bind a Windows directory to a Linux container is probably very painful but I do not want to do that.

I agree that figuring out how to upgrade the system while keeping the home directory as an external directory might be difficult. I wonder if it could be bound during the docker build process too.

Permissions should not be too much of an issue because I have a linux directory bound to linux container. We should not need to worry about permissions if the user IDs and group IDs are OK.

Having multiple binds and volumes sounds not cool but I can see how you end up there if binding the home is unstable.

I think group ID 1000 is missing from your build - I added this to my Dockerfile RUN groupadd -g 1000 headless

[accetto]

The group 1000 should be definitely there. You should see it if you'll run the test I've described above.

This part should be there:

+ tail -n2 /etc/group
messagebus:x:101:
headless:x:1000:

Also the following try inside the container will fail:

> groupadd -g 1000 headless
groupadd: group 'headless' already exists

Your scenario is of course simpler, then the cross-ones, but according my testing, sharing the whole $HOME folder will not work correctly even in your case, I'm afraid.

You can recognize it by comparing the desktop you will get to the screenshots I've published above. I would guess that you will not see the Version Sticker icon on you desktop.

[markNZed]

I'm not sure why I had to add that group - I should have got the error you saw.

I do see the Version Sticker on the desktop. But I don't understand the purpose of this.

What specifically (excluding updating of the base image) will not work with the whole $HOME shared?

[accetto]

According my testing, if you bind the whole $HOME directory to an external folder on the host, the container will only seemingly initialize correctly. You can recognize it easily by the missing Version Sticker on the desktop. It's no special thing otherwise. :) It simply starts the script, which displays the version sticker.

On my machines I have to initialize the container in some controlled environment first. It can be the container itself, without any bindings, or using a named Docker volume for the $HOME directory. The problem are some dot-files and/or dot-subfolders in the $HOME directory, but I had no time yet to find out which ones. However, you can bind any other parts of the $HOME directory.

You can also copy the complete $HOME content to some external directory after it has been fully initialized. Then you can bind that already initialized content. This is essentially what I've tried to describe above.

By containers not including GUI desktops you can usually bind the whole $HOME directory without big problems.

If you've really got the Version Sticker icon on your desktop, then it would be something new and actually a good news.

It could mean, that the container has initialized correctly in your environment. If it's so, we could dig out some helpful hints.

[markNZed]

There is no way to bind a directory during build so we have to copy the $HOME from the initial container into a host directory and then restart the container binding the $HOME.

It should be possible to copy the $HOME directory into the container when building later versions, this would allow for an upgrade path.

You wrote "Be sure to make the copy from inside the container. If you do it using the docker cp command, then not all files will be copied correctly" and I did use docker cp but I am not yet seeing problems. What problems a re caused by using docker cp ?

[markNZed]

One issue I've noticed is that if we want to copy the home directory during build then the directory structure needs to be something like:

/mnt/e/docker/headless

/mnt/e/docker/Dockerfile containing COPY headless /home

I was having an issue with trying to COPY the /mnt/e/docker/headless directory when running docker build from another directory

[accetto]

You wrote "Be sure to make the copy from inside the container. If you do it using the docker cp command, then not all files will be copied correctly" and I did use docker cp but I am not yet seeing problems. What problems a re caused by using docker cp ?

What I mean is, that docker cp didn't work correctly in at least one of the environments I use for testing.

There is also another point worth mentioning. If you copy the files inside the container, you don't need to care about the actual host environment configuration. You use the encapsulated folder tree inside the container and the pathes are always the same. This would simplify the possible helper scripts you could write. The pathes on the host are dependent on the current host environment configuration.

Generally speaking, any discussion about similar subjects is of very limited usefulness, unless we give a sufficient description of the environment and the conditions, under which some approach works or doesn't work. There is no way around defining good use and test cases first.

I can summarize what I've found till now this way:

1. I do not recommend using bind mounts for the whole $HOME directories with Docker containers that provide GUI desktops.
2. The only solution that works with this use case across the environments is using Docker volumes.
3. Generally I do not recommend persisting the whole $HOME directory outside the container, but to follow the application profiles approach instead. In such case you bind directories for different application separately. For example, Firefox profile, Visual Studio Code profile, etc.

[accetto]

if we want to copy the home directory during build

If we want to copy the home directory during the build time, then we have to be able to prepare all files that are required before we start building the image. In other words, there should be no files that must be created during the run time. This requirements is not always possible to fulfill. It's much simpler with the images that do not provide GUI desktops, of course. That is why I use a combined approach in my images. Some files are added during the build time (like some pre-configurations) and some are created during the run time. Then you have to care about the correct permissions and ownership, of course. And to be aware, that the user can override the container user and/or the user group.

Developing solutions for concrete use cases and environments is possible. Developing general solutions for all use cases and all environments is out of our reach.

The best answers till now are managed environments - Docker volumes and the clouds.
And also those really support only a limited subset of scenarios and mostly only in their own environment.

[accetto]

The new User guide contains a section about binding data to my containers. Examples and descriptions of pitfalls are also included. Therefore I'm closing this discussion. Please use the Issues of the User guide project for feedback.