Domino can automatically adapt most Docker images so that they are compatible for use with Domino Jobs and Workspaces. This functionality makes it easy to utilize external catalogues of pre-built images---like the GPU-optimized NVIDIA GPU Cloud (NGC) containers---inside Domino.

Usage

Create a new Environment by clicking Create Environment on the Environments overview page.
Select Start from a custom base image as the Base Environment and paste the container image URI into the text box.

Automatically make compatible with Domino should be checked by default, except when using a pre-built Domino image starting with 'quay.io/domino'. Leave it checked to use the functionality documented here.

See the Environment Management <Environment_management> docs for more on how to create an Environment.

How it works

When creating an environment using the automatic compatibility feature, Domino will:

Validate that these pre-requisites <compulsory_deps> are present in the image.
Add the dependencies required for use with Domino in an isolated location.
Build the environment.

At this point, the environment is a regular Domino Environment and can be used and edited as such.
The functionality of your image will remain untouched. Code and tools that runs using the image in Domino will work when using the image outside of Domino and vice versa.

Note

Pre-requisites for automatic custom image compatibility with Domino

There are certain pre-requisites that must be present for an image to be used with the automatic compatibility feature. There are also optional requirements that affect which Workspace Tools will work in your custom image environment.

Compulsory pre-requisites - required for automatic compatibility

The following requirements must be present in custom images for use with the automatic compatibility feature:

bash 4.2+
shadow-utils, including
- groupadd
- useradd
glibc 2.23+, including:
- libc.so.6
- libpthread.so.0
- libcrypt.so.1
- libutil.so.1
- libdl.so.2
- libm.so.6

These requirements are normally satisfied by images based on the following off-the-shelf distributions of Linux:

CentOS Linux 7 (or later)
Debian GNU/Linux 10 (or later)
Red Hat Enterprise Linux 8.2 (or later)
Ubuntu 16.04.2 LTS (or later)

Note

Optional pre-requisites - required for Domino Workspace Tools

The following table shows the requirements that must be present in the image for each of the default Domino Workspace Tools as well as the requirements for Domino Jobs. If a certain tool isn’t working, check to see if its requirements are present in the image. You can add any missing requirements by either modifying the base image itself or by adding Dockerfile instructions to the Domino Environment’s Dockerfile Instructions block.

Workspace Tool	Requires Python	Requires Jupyter
Domino Jobs	No	No
VS Code	Yes	No
JupyterLab (kernels present)	Yes	Yes
JupyterLab (terminal only)	No	No

Workspace Tool

Requires Python

Requires Jupyter

Domino Jobs

VS Code

Yes

JupyterLab (kernels present)

Yes

JupyterLab (terminal only)

Note the following:

Jobs can be used without Python or Jupyter. However, if you want to run a Python script in a job, then you need Python in the custom image.
JupyterLab can be launched even if Python and Jupyter are not installed, but only a basic terminal will be available and the JupyterLab "Launcher" will be empty. To use a terminal, open JupyterLab and then click File > New > Terminal.

Modify the default workspace tools

Caution

Note

To replace the default tools in a Domino environment (such as JupyterLab or VS Code) with the tools in your image, you can add a startup script for each tool to your environment, using one of these methods:

adding the script to the base image
adding it to the Dockerfile instructions

Follow the guidance below for the tool you want to modify:

VS Code

See Using Visual Studio Code in Domino Workspaces <../workspaces/Using_Visual_Studio_Code_in_Domino_Workspaces>.
JupyterLab

An example script below shows how the JupyterLab script should configure access to files in the environment, set up the working directory, and define the necessary routing.
Other tools

See Adding New Workspace Tools <Adding_new_workspaces_with_pluggable_notebooks>.

After adding the script to the environment, modify the Pluggable Workspace Tools section of the environment definition to define the entrypoint for workspace tool startup:

Go to the environment you want to edit.
Click Edit Definition.
Scroll down to Pluggable Workspace Tools.
Modify the desired entry so that the start field points to your tool’s launch script.

Example launch script for JupyterLab

BASE_URL="/$DOMINO_PROJECT_OWNER/$DOMINO_PROJECT_NAME/notebookSession/$DOMINO_RUN_ID"
CONF_FILE="$HOME/.jupyter/jupyter_lab_config.py"
mkdir -p $(dirname "$CONF_FILE")

cat > "$CONF_FILE" << EOF
c = get_config()

# Lab has access to all files in the environment
c.NotebookApp.notebook_dir='/'

# Lab starts in the domino working dir
c.NotebookApp.default_url='/lab/tree${DOMINO_WORKING_DIR}'

# Routing, networking and access
c.NotebookApp.base_url='${BASE_URL}/'
c.NotebookApp.tornado_settings={'headers': {'Content-Security-Policy': 'frame-ancestors *'}, 'static_url_prefix': '${BASE_URL}/static/'}
c.NotebookApp.token=u''
c.NotebookApp.iopub_data_rate_limit=10000000000

# The default cell execution timeout in nbconvert is 30 seconds, set it to a year
c.ExecutePreprocessor.timeout=365*24*60*60

EOF

Manually modify images for Domino compatibility

Caution

When you create an environment with the automatic compatibility feature enabled, Domino adds a number of Dockerfile commands so that the custom images can launch Job and Workspace containers. At a high level, these instructions perform the following actions:

Create a non-root Domino user
Install sudo and curl

Sudo and curl installations are required for these images to work in Domino. Curl is used for transferring data, while sudo enables users to install additional packages and libraries in the container, depending on the permissions set up by your Domino administrator.
Add an isolated directory, /opt/domino/, with the tools required for running Domino Workspaces.

The Dockerfile instructions below show you how to replicate these steps when building a custom image.

# Ensure you are injecting as the root user
USER root

# Add a Domino user and group ID
ARG DOMINO_USER='domino'
ARG DOMINO_GROUP='domino'

# The variable names that can be used in generation of environment scripts
ARG ALLENV="\$INSTALL_DIR,\$INSTALL_BIN,\$CONDA_DIR,\$DOMINO_UID,\$DOMINO_GID,\$DOMINO_USER,\$DOMINO_GROUP,\$VSCODE_DIR,\$VSCODE_EXT_DIR"

RUN \
  groupadd -g 12574 ${DOMINO_GROUP} && \
  useradd -u 12574 -g 12574 -m -N -s /bin/bash ${DOMINO_USER};

# Install common dependencies for the compiler and setting things up
ARG INSTALL_DIR=/opt/domino
ARG INSTALL_BIN=${INSTALL_DIR}/bin
ARG CONDA_DIR=${INSTALL_DIR}/conda

RUN \
  apt-get update && \
  apt-get -y install \
      build-essential \
      gettext-base \
      gnupg && \
  apt-get clean && \
  rm -rf /var/lib/apt/lists/* && \
  mkdir -p \
      ${INSTALL_DIR} \
      ${INSTALL_BIN} \
      ${CONDA_DIR}

# Recommended: Add domino user account to sudoers
RUN echo "${DOMINO_USER}    ALL=NOPASSWD: ALL" >> /etc/sudoers

# Set to a Domino supported language to prevent unrecognized character input
RUN echo "export LANG=${LANG}" >> /home/${DOMINO_USER}/.domino-defaults && \
    echo "export LC_ALL=${LANG}" >> /home/${DOMINO_USER}/.domino-defaults && \
    # Needed for proper filename handling in python
    echo 'export PYTHONIOENCODING=utf-8' >> /home/${DOMINO_USER}/.domino-defaults && \
    chown -R ${DOMINO_USER}.${DOMINO_GROUP} "/home/${DOMINO_USER}"

# load Domino defaults
RUN if [ -f /home/${DOMINO_USER}/.domino-defaults ]; then \
        echo "source /home/${DOMINO_USER}/.domino-defaults" >> /home/${DOMINO_USER}/.bashrc; \
    fi

# Install Sudo and Curl
# Note that this will only work with Debian and Ubuntu
RUN apt update && apt install -y --no-install-recommends \
        curl \
        sudo \
    # clean up
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

Optionally, if you want JupyterLab or any IDEs in this workspace, then define a start script for each. See the above section on how to modify the default workspace tools for an example. These start scripts are stored in /opt/domino/workspaces by default. Make sure to add a new field to your environment’s Pluggable Workspace Tools for the associated tool and set the start field to the location of your start script.

Optional: Enable JupyterLab

If you would like to use JupyterLab in a Domino workspace that uses a custom image, use the instructions below to install with miniconda. The commands for installing Tensorboard are included, and the comments indicate which lines to remove if you do not need Tensorboard in your image.

# Install JupyterLab and Tensorboard using Conda

# A proper combination of these versions is important!
ARG CONDA_URL=https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
ARG PYTHON_VERSION=3.9
ARG JUPYTER_VERSION=2.3
# remove the following line if you are not installing Tensorboard:
ARG TENSORBOARD_VERSION=2.2

# You must create a start script with configurations for JupyterLab
ARG JUPYTER_SCRIPT=jupyterlab-start.sh

# Install Conda and Python
# (Download signature not available)
RUN \
  curl -o conda-install.sh -LSsf ${CONDA_URL} && \
  /bin/bash conda-install.sh -fbp ${CONDA_DIR} && \
  ${CONDA_DIR}/bin/conda update -y --update-all && \
  ${CONDA_DIR}/bin/conda install -y python=${PYTHON_VERSION}

# Install NodeJS if installing Tensorboard
RUN \
  ${CONDA_DIR}/bin/conda install -y nodejs && \
  mkdir -p ${INSTALL_DIR}/node && \
  ln -rs ${CONDA_DIR}/bin/node ${INSTALL_DIR}/node/ && \
  ln -rs ${CONDA_DIR}/bin/npm ${INSTALL_DIR}/node/

# Install and configure Jupyterlab and extensions
RUN \
  ${CONDA_DIR}/bin/conda install -y jupyterlab=${JUPYTER_VERSION} && \
  # remove the following lines if you are not installing Tensorboard:
  ${CONDA_DIR}/bin/conda install -y tensorboard=${TENSORBOARD_VERSION} && \
  ${CONDA_DIR}/bin/conda install -y -c conda-forge jupytext && \
  ${CONDA_DIR}/bin/conda install -y nodejs && \
  ${CONDA_DIR}/bin/pip install jupyter_tensorboard && \
  PATH=${CONDA_DIR}/bin ${CONDA_DIR}/bin/jupyter labextension install jupyterlab_tensorboard

# Create Juptyerlab workspace environment scripts
RUN \
  envsubst "${ALLENV}" < ${JUPYTER_SCRIPT} > ${INSTALL_BIN}/${JUPYTER_SCRIPT} && \
  chmod 755 ${INSTALL_BIN}/${JUPYTER_SCRIPT}

Optional: Enable VS Code

To use VS Code in a Domino workspace, add the instructions below.

# Install VS Code

#Set Versions
ARG VSCODE_VERSION=3.10.2
ARG VSCODE_URL=https://github.com/cdr/code-server/releases/download/v${VSCODE_VERSION}/code-server-${VSCODE_VERSION}-linux-amd64.tar.gz
ARG VSCODE_PYTHON_VERSION=2021.5.926500501
ARG VSCODE_PYTHON_URL=https://github.com/microsoft/vscode-python/releases/download/${VSCODE_PYTHON_VERSION}/ms-python-release.vsix

# Where it will be deployed
ARG VSCODE_DIR=${INSTALL_DIR}/vscode
ARG VSCODE_EXT_DIR=${VSCODE_DIR}/extensions

# You must create a start script with configurations for VS Code
ARG VSCODE_SCRIPT=vscode-start.sh

RUN \
  curl -LSsf ${VSCODE_URL} | tar -xz --no-same-permissions && \
  mv -f code-server-${VSCODE_VERSION}-linux-amd64 ${VSCODE_DIR} && \
  curl -o python.vsix -LSsf ${VSCODE_PYTHON_URL} && \
  mkdir -p ${VSCODE_EXT_DIR} && \
  ${VSCODE_DIR}/bin/code-server \
      --install-extension python.vsix \
      --extensions-dir ${VSCODE_EXT_DIR}

# Create VS Code workspace environment scripts
RUN \
  envsubst "${ALLENV}" < ${VSCODE_SCRIPT} > ${INSTALL_BIN}/${VSCODE_SCRIPT} && \
  chmod 755 ${INSTALL_BIN}/${VSCODE_SCRIPT}

Automatic Adaptation of Custom Images

Usage

How it works

Pre-requisites for automatic custom image compatibility with Domino

Modify the default workspace tools

Example launch script for JupyterLab

Manually modify images for Domino compatibility

Optional: Enable JupyterLab

Optional: Enable VS Code