You can create new Domino environments and edit existing environments to meet your language and package needs. You can create or modify an environment when:
-
You must install a package for Python, R, Octave, or some other software dependency.
-
You want to cache a package or library so that it’s immediately available when you start a run.
-
You are managing an organization, and want to create a default environment across all projects.
A Domino environment is an abstraction on top of a Docker image. When Domino starts your run, it creates an isolated Docker container based on the environment associated with your project. This helps provide flexibility and versioning for your runs.
-
Go to the project for which you want to set a default environment. In the navigation pane, click Settings.
-
From Compute environment, select the environment.
-
Click Manage Environment to open the environment’s Overview tab.
In the navigation pane, click Environments to see the environments to which you have access, including:
-
Your deployment’s global environments.
-
Environments used by projects with which you are a collaborating.
-
Environments shared with organizations to which you belong.
Create an environment
-
Click Create Environment.
-
Name your environment and give it a clear Description.
-
Set the Base Environment / Image. You can base your compute environment on an existing environment or on a custom Dockerfile URI (for example,
registry.hub.docker.com/library/python:3.8-slim
). If you use a custom image, you must define the FROM line in the Dockerfile that Domino constructs for you. -
Select the Supported Clusters.
-
Select the environment’s Visibility. Administrators can make the environment Globally Accessible, which makes it available to all projects.
-
Click Create Environment. The Revisions tab for the environment opens. If you want to customize the environment by editing the Dockerfile instructions, click Customize before Building and see the following procedure.
Customize a new environment
-
In Dockerfile Instructions, enter your Dockerfile layers. See Docker’s official site. You can also view the Dockerfile best practices.
-
In Pluggable Workspace Tools, enter the interactive tools needed by any project that uses this environment. See Add Workspace IDEs.
-
In the Run Setup Scripts section, you can enter bash code that is executed at runtime. Domino executes these commands at runtime and runs them as root.
-
In Pre Run Script, enter commands to be run before the Python packages in your project’s
requirements.txt
are installed. -
In Post Run Script, enter commands to be run after the
requirements.txt
installation process.
-
-
In the Environment variables section, set the Variable Names and their Values.
-
Click Advanced if you want to enter bash code that is executed at a specified step in your experiment’s lifecycle.
-
In Pre Setup Script, enter commands to execute before the Python packages in your project’s
requirements.txt
are installed. -
In Post Setup Script, enter commands to execute after the Python packages in the
requirements.txt
of your project are installed.
-
-
In Revision History, enter a short description about the changes to the environment.
-
Click Build.
Modify an existing environment
-
In the navigation pane, click Environments.
-
Click the name of an environment.
-
On the Overview tab, you can modify the description or visibility.
-
Click Edit Definition. See Create an environment and Customize a new environment for details. Each time you save changes, the environment’s revision increments by one and your Domino deployment rebuilds the environment and pushes it to the local Docker registry.
View environment revisions
Each time you edit an environment, you create a new revision. The latest successfully built revision becomes active by default.New workspaces launch with this Active environment revision.
-
In the navigation pane, click Environments.
-
Click the name of an environment.
-
Click the Revisions tab. You can see all revisions of your compute environment, each revision’s build status, timestamp, and Docker image URI.
-
Click the three vertical dots icon at the end of a revision’s row to perform additional actions such as canceling builds.
You might want to install packages directly to your environment. This is useful if your package installation takes a long time. Installations in a Domino environment are cached, so you won’t have to wait for the package to install every time. If you already have a Docker image you’d like to use, enter it in the preceding Base Environment field. If you don’t set this, the Domino default environment is used as your base image.
Consult the official Docker documentation to learn more about Dockerfiles:
Do not start your Dockerfile instructions in Domino with a FROM
line. Domino includes the FROM
line for you, pointing to the base image specified when setting up the environment.
The most common Dockerfile instructions you’ll use are RUN, ENV, and ARG:
RUN
commands execute lines of bash. For example:
USER root
RUN wget http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz
RUN tar xvzf spark-1.5.1-bin-hadoop2.6.tgz
RUN mv spark-1.5.1-bin-hadoop2.6 /opt
RUN rm spark-1.5.1-bin-hadoop2.6.tgz
USER ubuntu
ARG
commands set build-time variables, and
ENV
commands set
container bash environment variables. They will be accessible from runs
that use this environment. For example:
ENV SPARK_HOME /opt/spark-1.5.1-bin-hadoop2.6
If you set variables in the Environment variables section of your definition, you can use an ARG
statement:
ARG SPARK_HOME
This will be available for the build step. If you want the variable to be available in the final compute environment you also need to add an ENV statement referencing the argument name:
ENV SPARK_HOME=$SPARK_HOME
Examples: Package Installation
Click R Package or Python Package when editing your environment, to insert a line to install packages. Enter the names of the packages.
You can also add the commands, as in the following examples:
-
R Package Installation, example with the devtools package.
USER root RUN R --no-save -e "install.packages('devtools')" USER ubuntu
-
Python Package Installation with Pip: Example with the numpy package.
USER root
RUN pip install numpy
USER ubuntu
-
Docker optimizes its build process by keeping track of commands it has run and aggressively caching the results. This means that if it sees the same set of commands as in a previous build, it will assume that it can use the cached version. A single new command will invalidate the caching of all subsequent commands.
-
A Docker image can have up to 127 layers, or commands. To work around this limit, you can use
&&
: to combine several commands into a single command.RUN \ wget http://d3kbcqa49mib13.cloudfront.net/spark-1.5.1-bin-hadoop2.6.tgz && \ tar xvzf spark-1.5.1-bin-hadoop2.6.tgz && \ mv spark-1.5.1-bin-hadoop2.6 /opt && \ rm spark-1.5.1-bin-hadoop2.6.tgz
-
If you are installing multiple python packages via pip, it’s almost always best to use a single pip install command. This ensures that dependencies and package versions are properly resolved. If you install via separate commands, you may end up inadvertently overriding a package with the wrong version, due a dependency specified by a later installation. For example:
RUN pip install luigi nolearn lasagne