Container engine initialization error, unclear why

[startakovsky]

Bug description

I ran this command: jupyter-repo2docker . and apparently it cannot find the docker version. Even when I run it in debug mode I get no real info ([Repo2Docker] Looking for repo2docker_config in #..., and when I search the code for this string repo2docker_config in the documentation and code nothing really comes up.

Expected behaviour

I thought something would be built, because I am following this documentation: https://repo2docker.readthedocs.io/en/latest/usage.html#calling-repo2docker

Actual behaviour

Container engine initialization error: ('Check if docker is running on the host.', DockerException("Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))"))

How to reproduce

1. Buy a mac
2. Install brew
3. Install pyenv
4. Do pyenv install 3.11.3
5. Do pyenv global 3.11.3
6. Do pip install pipx
7. pipx install jupyter-repo2docker
8. Navigate to a directory where you want to build a docker image.
9. Install Docker, Docker version 24.0.2, build cb74dfc
10. Do jupyter-repo2docker .

Your personal set up

• OS: OSX 13.3.1
• Docker version: Docker version 24.0.2, build cb74dfc
• repo2docker version 2023.06.0

What happens when I run jupyter-repo2docker --no-build .

FROM docker.io/library/buildpack-deps:bionic

# Avoid prompts from apt
ENV DEBIAN_FRONTEND=noninteractive

# Set up locales properly
RUN apt-get -qq update && \
    apt-get -qq install --yes --no-install-recommends locales > /dev/null && \
    apt-get -qq purge && \
    apt-get -qq clean && \
    rm -rf /var/lib/apt/lists/*

RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen && \
    locale-gen

ENV LC_ALL=en_US.UTF-8 \
    LANG=en_US.UTF-8 \
    LANGUAGE=en_US.UTF-8

# Use bash as default shell, rather than sh
ENV SHELL=/bin/bash

# Set up user
ARG NB_USER
ARG NB_UID
ENV USER=${NB_USER} \
    HOME=/home/${NB_USER}

RUN groupadd \
        --gid ${NB_UID} \
        ${NB_USER} && \
    useradd \
        --comment "Default user" \
        --create-home \
        --gid ${NB_UID} \
        --no-log-init \
        --shell /bin/bash \
        --uid ${NB_UID} \
        ${NB_USER}

# Base package installs are not super interesting to users, so hide their outputs
# If install fails for some reason, errors will still be printed
RUN apt-get -qq update && \
    apt-get -qq install --yes --no-install-recommends \
       gettext-base \
       less \
       unzip \
       > /dev/null && \
    apt-get -qq purge && \
    apt-get -qq clean && \
    rm -rf /var/lib/apt/lists/*

EXPOSE 8888

# Environment variables required for build
ENV APP_BASE=/srv
ENV CONDA_DIR=${APP_BASE}/conda
ENV NB_PYTHON_PREFIX=${CONDA_DIR}/envs/notebook
ENV NPM_DIR=${APP_BASE}/npm
ENV NPM_CONFIG_GLOBALCONFIG=${NPM_DIR}/npmrc
ENV NB_ENVIRONMENT_FILE=/tmp/env/environment.lock
ENV MAMBA_ROOT_PREFIX=${CONDA_DIR}
ENV MAMBA_EXE=${CONDA_DIR}/bin/mamba
ENV CONDA_PLATFORM=linux-aarch64
ENV KERNEL_PYTHON_PREFIX=${NB_PYTHON_PREFIX}
# Special case PATH
ENV PATH=${NB_PYTHON_PREFIX}/bin:${CONDA_DIR}/bin:${NPM_DIR}/bin:${PATH}
# If scripts required during build are present, copy them

COPY --chown=501:501 build_script_files/-2fusers-2fsteven-2f-2elocal-2fpipx-2fvenvs-2fjupyter-2drepo2docker-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2factivate-2dconda-2esh-26bc03 /etc/profile.d/activate-conda.sh

COPY --chown=501:501 build_script_files/-2fusers-2fsteven-2f-2elocal-2fpipx-2fvenvs-2fjupyter-2drepo2docker-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2fenvironment-2epy-2d3-2e10-2dlinux-2daarch64-2elock-9c3286 /tmp/env/environment.lock

COPY --chown=501:501 build_script_files/-2fusers-2fsteven-2f-2elocal-2fpipx-2fvenvs-2fjupyter-2drepo2docker-2flib-2fpython3-2e11-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2finstall-2dbase-2denv-2ebash-a3eaa9 /tmp/install-base-env.bash
RUN TIMEFORMAT='time: %3R' \
bash -c 'time /tmp/install-base-env.bash' && \
rm -rf /tmp/install-base-env.bash /tmp/env

RUN mkdir -p ${NPM_DIR} && \
chown -R ${NB_USER}:${NB_USER} ${NPM_DIR}


# ensure root user after build scripts
USER root

# Allow target path repo is cloned to be configurable
ARG REPO_DIR=${HOME}
ENV REPO_DIR=${REPO_DIR}
# Create a folder and grant the user permissions if it doesn't exist
RUN if [ ! -d "${REPO_DIR}" ]; then \
        /usr/bin/install -o ${NB_USER} -g ${NB_USER} -d "${REPO_DIR}"; \
    fi

WORKDIR ${REPO_DIR}
RUN chown ${NB_USER}:${NB_USER} ${REPO_DIR}

# We want to allow two things:
#   1. If there's a .local/bin directory in the repo, things there
#      should automatically be in path
#   2. postBuild and users should be able to install things into ~/.local/bin
#      and have them be automatically in path
#
# The XDG standard suggests ~/.local/bin as the path for local user-specific
# installs. See https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
ENV PATH=${HOME}/.local/bin:${REPO_DIR}/.local/bin:${PATH}

# The rest of the environment
ENV CONDA_DEFAULT_ENV=${KERNEL_PYTHON_PREFIX}
# Run pre-assemble scripts! These are instructions that depend on the content
# of the repository but don't access any files in the repository. By executing
# them before copying the repository itself we can cache these steps. For
# example installing APT packages.
# If scripts required during build are present, copy them

COPY --chown=501:501 src/binder/environment.yml ${REPO_DIR}/binder/environment.yml
USER ${NB_USER}
RUN TIMEFORMAT='time: %3R' \
bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file "binder/environment.yml" && \
time ${MAMBA_EXE} clean --all -f -y && \
${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} \
'


# ensure root user after preassemble scripts
USER root

# Copy stuff.
COPY --chown=501:501 src/ ${REPO_DIR}/

# Run assemble scripts! These will actually turn the specification
# in the repository into an image.


# Container image Labels!
# Put these at the end, since we don't want to rebuild everything
# when these change! Did I mention I hate Dockerfile cache semantics?

LABEL repo2docker.ref="None"
LABEL repo2docker.repo="local"
LABEL repo2docker.version="2023.06.0"

# We always want containers to run as non-root
USER ${NB_USER}

# Make sure that postBuild scripts are marked executable before executing them
RUN chmod +x binder/postBuild
RUN ./binder/postBuild

# Add start script
# Add entrypoint
ENV PYTHONUNBUFFERED=1
COPY /python3-login /usr/local/bin/python3-login
COPY /repo2docker-entrypoint /usr/local/bin/repo2docker-entrypoint
ENTRYPOINT ["/usr/local/bin/repo2docker-entrypoint"]

# Specify the default command to run
CMD ["jupyter", "notebook", "--ip", "0.0.0.0"]

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[startakovsky]

Docker Desktop

[startakovsky]

Has anyone seen this?

[minrk]

I haven't, but it suggests that the docker API socket isn't where it usually is. Do you have an docker env set (env | grep DOCKER)? And does docker ps, etc. work as you'd expect?

[startakovsky]

@minrk I do not have any docker env set, and docker ps works as expected. Thanks.

[luzhengyang]

I have the exact same error and no idea why this happens. docker is working as expected.

[hiroyuki-sato]

Setting the environment variable DOCKER_HOST may solve this issue?

With DOCKER_HOST

export DOCKER_HOST=unix:///Users/user/.docker/run/docker.sock

(I found this path using netstat -na | grep docker | awk '{ print $NF }' | sort -u)

jupyter-repo2docker https://github.com/norvig/pytudes
Picked Git content provider.
Cloning into '/var/folders/c4/n95yngmn1kz96x7jslsnqjr80000gn/T/repo2dockeru7mzl4c7'...
remote: Enumerating objects: 256, done.
remote: Counting objects: 100% (256/256), done.
remote: Compressing objects: 100% (250/250
...

Without DOCKER_HOST

jupyter-repo2docker https://github.com/norvig/pytudes

Container engine initialization error: ('Check if docker is running on the host.', DockerException("Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))"))

My environment.

• Python: 3.10.11
• jupyter-repo2docker: 2023.06.0

sw_vers
ProductName:		macOS
ProductVersion:		13.5.2
BuildVersion:		22G91

docker version
Client: Docker Engine - Community
 Version:           23.0.1
 API version:       1.42
 Go version:        go1.19.5
 Git commit:        a5ee5b1dfc
 Built:             Thu Feb  9 19:15:59 2023
 OS/Arch:           darwin/arm64
 Context:           default

Server: Docker Desktop 4.24.2 (124339)
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:31:36 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[felder]

I'm also seeing this behavior on MacOS 13.6, with jupyter-repo2docker 2023.06.0, python 3.9.18, and Docker Desktop 4.24.2

[hiroyuki-sato]

Hello, @felder. Have you ever tried DOCKER_HOST environment variable?
#1289 (comment)

[felder]

@hiroyuki-sato yes, that seems to fix it. Thanks!

[felder]

I found a fix for this that does not require exporting DOCKER_HOST.

Open Docker Destop and go to Settings -> Advanced -> Check "Allow the default Docker socket to be used"

After Docker Desktop restarts, repo2docker works as normal!

[mathieuboudreau]

I found a fix for this that does not require exporting DOCKER_HOST.

Open Docker Destop and go to Settings -> Advanced -> Check "Allow the default Docker socket to be used"

After Docker Desktop restarts, repo2docker works as normal!

Worked for me - thanks!

[lauramariel]

Does anyone know if this works with Rancher Desktop? I tried the workaround and it didn't work for me, unfortunately.

Container engine initialization error: ('Check if docker is running on the host.', DockerException('Invalid bind address format: /Users/laura/.rd/docker.sock'))

Nevermind, forgot to prefix with unix:///. Works now!