Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

additional runners that we could look at including #72

Open
tomeichlersmith opened this issue Dec 29, 2023 · 2 comments
Open

additional runners that we could look at including #72

tomeichlersmith opened this issue Dec 29, 2023 · 2 comments

Comments

@tomeichlersmith
Copy link
Owner

I haven't done a very exhaustive search, but I have stumbled upon other runners that may be useful for denv to support.

  • lilipod is a container manager focused on pulling images, managing images, and running containers. It was created as a simple/light backend for distrobox and so it naturally works as a runner for denv as well.
  • charliecloud is a container manager focused on bringing easy-to-manage containers to HPCs. This manager is appealing to me since I could see it ending up on HPCs due to its security-focused design similar to singularity/apptainer.
@tomeichlersmith
Copy link
Owner Author

tomeichlersmith commented Dec 30, 2023

Did some charliecloud investigation, here are my notes

  • need to use --write with ch-run so we can create the destination directories when --bind
    • we could avoid --write and use the pre-created /mnt/[0-9] but then users would need to get used to the paths changing within the denv which I'd like to avoid
    • I actually kinda like this explicit-ness for mounting from the host into the container, but it does make the containerized environment more difficult to use
  • need to create a dir formatted image so we can use --write
  • can't use --home because that tries to --bind ${workspace}:/home/$USER in the container which is not what we want (and fails if /home/$USER doesn't exist in the image)
  • --bind and --set-env allows us to do the mounting and environment setup as need be

In a simple ubuntu server VM.

wget -q -O - https://github.com/hpc/charliecloud/releases/download/v0.35/charliecloud-0.35.tar.gz \
  | tar xzf -
cd charliecloud-0.35
./configure.sh
make
sudo make install
cd examples/hello
ch-image build . # infers name as name of directory
cd ../..
mkdir tutorial
cd tutorial
ch-convert -o dir hello hello
ch-run --write --cd=${PWD} --bind ${PWD} --set-env=HOME=${PWD} -- /bin/bash

@tomeichlersmith
Copy link
Owner Author

I can't get squashfuse internal mounting to work on Ubuntu because charliecloud requires libfuse3 while the squashfuse installable on Ubuntu 22.04 uses libfuse2.

tom@ch-playground:~/charliecloud-0.35$ sudo apt install libsquashfuse-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  libfuse2 libsquashfuse0
The following NEW packages will be installed:
  libfuse2 libsquashfuse-dev libsquashfuse0
0 upgraded, 3 newly installed, 0 to remove and 45 not upgraded.
Need to get 146 kB of archives.
After this operation, 628 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

I'm unsure on how to proceed from here although I suppose I could try to roll my own build of squashfuse.

@tomeichlersmith tomeichlersmith changed the title addition runners that we could look at including additional runners that we could look at including Jan 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant