Each run and workspace in Domino operates in its own Docker container. These Docker containers are defined by Domino environments. Environments can be shared and customized, and they are automatically versioned by Domino.
New installations of Domino come with a standard set of environments and associated Docker images. Periodically, Domino publishes a new set of standard environments with updated libraries and packages. This topic describes how to adopt the latest Domino standard environments.
Domino maintains three standard compute environments:
-
Domino Analytics Distribution (DAD) for Python 2.7
-
Domino Analytics Distribution (DAD) for Python 3.6
-
Domino Analytics Distribution (DAD) for Python 3.7
The Domino Analytics Distributions are designed to handle most of what a typical data science workflow needs out of the box, and differ only on which version of Python they have installed.
These images are hosted on Quay.io, a third-party Docker registry. By default, your Domino deployment comes configured with keys to access Domino’s images in this service.
The URLs for the latest versions of Domino Standard Environments are:
-
DAD Python 2.7
quay.io/domino/base:Ubuntu18_DAD_Py2.7_R3.5-20190501 -
DAD Python 3.6
quay.io/domino/base:Ubuntu18_DAD_Py3.6_R3.5-20190501 -
DAD Python 3.7
quay.io/domino/base:Ubuntu18_DAD_Py3.7_R3.5-20190501
-
Ubuntu 18.04
-
Anaconda Python 2.7, 3.6 or 3.7
-
R 3.5.3
-
Jupyter Lab 0.35.4
-
Jupyter 1.0
-
Rstudio 1.2
-
VSCode
-
Keras 2.2.4 and Tensorflow-gpu* 1.13.1
-
CUDA 10.0.130 with NVIDIA driver 410.104
-
Open JDK 8
-
Scala 2.12.6
-
Julia 1.1
-
Node 8.10
-
Various common utilities (Curl, vim, AWSCLI, telnet, etc)
See here for a full list of python and R packages:
Note: to use Tensorflow on a cpu machine you must install Tensorflow’s cpu version (e.g. pip install tensorflow).
To determine which version of Domino standard environments your deployment is using, open Domino and click Environments in the main menu.
You can use the search bar to find environments with Domino Analytics in the name.
However, previous administrators or installers may have added these environments under a different name.
To confirm that an environment is using a Domino standard, click on it to open the Overview.
|
Note
|
Your installation of Domino may be set up with a previous version of our standard base image. Examples of historical base images:
You may also have a custom base image with your companies name in the URL. If you are using one of these older images, you may require an upgrade to your system before you are able to use the Ubuntu 16.04 or 18.04 images first released in August 2018. |
Adding a Domino Standard Environment to your deployment involves four high-level steps.
-
Create a new environment with the correct defaults for the standard you want to add.
-
See Compute Environment Management for an overview of how to create and manage environments.
-
See the table of environment details below to populate the Base Image, Dockerfile Instructions, and Pluggable Workspace Tools fields. These fields must be filled out correctly for the environment to work as intended.
-
Critical Note: To use the latest version of the base image, you must have the Pluggable Workspace Tools feature turned on.
-
-
Copy over any additional customization from your existing default compute environment.
-
If you added additional packages and libraries to your previous default environment, and you intend to use the new standard environment as your deployment default, you should copy over those additions to the new environment you created. These can be added to the end of the Dockerfile Instructions in the new environment.
-
You should not copy over any instructions that update Python, R, RStudio, or other core components of the standard environments if the new standard already includes the version you want.
-
Be sure to copy over any setup scripts and run scripts attached to your previous default, as your users may depend on them.
-
-
Update the NVIDIA drivers on your executor template (AWS customers using GPUs only) or update the Nvidia on your GPU executor (On-Premises customers using GPUs only)
-
Only all other changes you make to a compute environment when using packages optimized for GPUs (e.g. Tensorflow-gpu and CUDA) there is a dependency between the host machine and the container. In this case the nvidia drivers must match.
-
Please reach out to support@dominodatalab.com
-
-
Create a new machine image for your executor template that includes the new environment. (AWS customers only)
-
See Creating a new executor AMI for instructions.
-
Once this is done, you can set the new environment as the deployment default.
-
You may also choose to archive the previous default environment. Any existing projects using an archived environment will be able to continue using it, but new projects will not have access to it.
-
This section contains detailed specifications about the two Domino Standard Environments, plus the configuration settings needed when adding them to your deployment.
Domino Analytics Distribution (DAD) for Python 2
-
Title:
Domino Analytics Distribution py2.7 R3.5
-
Description:
Ubuntu 18.04 Anaconda Python 2.7 R 3.5.3 Jupyter Lab 0.35.4, Jupyter 1.0, Rstudio 1.2, VSCode Keras 2.2.4 and Tensorflow-gpu* 1.13.1 CUDA 10.0.130 with NVIDIA driver 410.104 For python packages run: !pip freeze For R packages run: installed.packages() For further detail, ask Domino Support for the full Dockerfile -
Base Image URL:
quay.io/domino/base:Ubuntu18_DAD_Py2.7_R3.5-20190501 -
Pluggable Workspace Tools
jupyterlab: title: "JupyterLab (Beta)" iconUrl: "/assets/images/workspace-logos/jupyterlab.svg" start: [ /var/opt/workspaces/Jupyterlab/start.sh ] httpProxy: internalPath: "/{{ port: 8888 rewrite: false rstudio: title: "RStudio" iconUrl: "/assets/images/workspace-logos/Rstudio.svg" start: [ "/var/opt/workspaces/rstudio/start" ] httpProxy: port: 8888 vscode: title: "vscode" iconUrl: "/assets/images/workspace-logos/vscode.svg" start: [ "/var/opt/workspaces/vscode/start" ] httpProxy: port: 8888 jupyter: title: "Jupyter (Python, R, Julia)" iconUrl: "/assets/images/workspace-logos/Jupyter.svg" start: [ "/var/opt/workspaces/jupyter/start" ] httpProxy: port: 8888 rewrite: false internalPath: "/{{ supportedFileExtensions: [ ".ipynb" ]
Domino Analytics Distribution (DAD) for Python 3.6
-
Title:
Domino Analytics Distribution py3.6 R3.5 -
Description:
Ubuntu 18.04 Anaconda Python 3.6 R 3.5.3 Jupyter Lab 0.35.4 ,Jupyter 1.0 ,Rstudio 1.2, VSCode Keras 2.2.4 and Tensorflow-gpu* 1.13.1 CUDA 10.0.130 with NVIDIA driver 410.104 For python packages run: !pip freeze For R packages run: installed.packages() For further detail, please ask Domino Support for the full Dockerfile -
Base Image URL:
quay.io/domino/base:Ubuntu18_DAD_Py3.6_R3.5-20190501 -
Pluggable Workspace Tools
jupyterlab: title: "JupyterLab (Beta)" iconUrl: "/assets/images/workspace-logos/jupyterlab.svg" start: [ /var/opt/workspaces/Jupyterlab/start.sh ] httpProxy: internalPath: "/{{ port: 8888 rewrite: false rstudio: title: "RStudio" iconUrl: "/assets/images/workspace-logos/Rstudio.svg" start: [ "/var/opt/workspaces/rstudio/start" ] httpProxy: port: 8888 vscode: title: "vscode" iconUrl: "/assets/images/workspace-logos/vscode.svg" start: [ "/var/opt/workspaces/vscode/start" ] httpProxy: port: 8888 jupyter: title: "Jupyter (Python, R, Julia)" iconUrl: "/assets/images/workspace-logos/Jupyter.svg" start: [ "/var/opt/workspaces/jupyter/start" ] httpProxy: port: 8888 rewrite: false internalPath: "/{{ supportedFileExtensions: [ ".ipynb" ]
Domino Analytics Distribution (DAD) for Python 3.7
-
Title:
Domino Analytics Distribution py3.7 R3.5 -
Description:
Ubuntu 18.04 Anaconda Python 3.7 R 3.5.3 Jupyter Lab 0.35.4, Jupyter 1.0, Rstudio 1.2, VSCode Keras 2.2.4 and Tensorflow-gpu* 1.13.1 CUDA 10.0.130 with NVIDIA driver 410.104 For python packages run: !pip freeze For R packages run: installed.packages() For further detail, please ask Domino Support for the full Dockerfile -
Base Image URL:
quay.io/domino/base:Ubuntu18_DAD_Py3.7_R3.5-20190501 -
Pluggable Workspace Tools
jupyterlab: title: "JupyterLab (Beta)" iconUrl: "/assets/images/workspace-logos/jupyterlab.svg" start: [ /var/opt/workspaces/Jupyterlab/start.sh ] httpProxy: internalPath: "/{{ port: 8888 rewrite: false rstudio: title: "RStudio" iconUrl: "/assets/images/workspace-logos/Rstudio.svg" start: [ "/var/opt/workspaces/rstudio/start" ] httpProxy: port: 8888 vscode: title: "vscode" iconUrl: "/assets/images/workspace-logos/vscode.svg" start: [ "/var/opt/workspaces/vscode/start" ] httpProxy: port: 8888 jupyter: title: "Jupyter (Python, R, Julia)" iconUrl: "/assets/images/workspace-logos/Jupyter.svg" start: [ "/var/opt/workspaces/jupyter/start" ] httpProxy: port: 8888 rewrite: false internalPath: "/{{ supportedFileExtensions: [ ".ipynb" ]