Use requirements.txt (Python only)

Unless package persistence is enabled, Domino runs each script or interactive session in a fresh environment. In this case, any custom packages are reinstalled at the start of each execution.

To add Python dependencies to your project:

  • Create a requirements.txt file in the root of your project directory.

  • List the required packages using standard pip syntax.

Domino will install these packages automatically at runtime.

Domino provides flexibility, so if you want your changes to remain permanently installed, see Preload Environment packages to create a custom environment.

Many common modules are installed by default. To get a list of pre-installed modules, include help('modules') at the start of your script or in a new IPython Notebook session. You can also run domino run --direct "pip freeze" with the Domino CLI tool.

Improve execution startup time

Using a requirements.txt file allows you to specify and install Python dependencies at runtime. However, this approach comes with trade-offs:

  • For each execution, Domino installs the listed packages from scratch, which can increase startup time.

To optimize performance based on your use case:

  • For Workspaces: Enable package persistence to install packages once during initial startup. These packages will then be retained between sessions, avoiding repeat installations and significantly reducing startup time for future launches.

  • For Jobs, Apps, and other executions: Use preloaded Environment packages to build a custom environment where packages are baked into the image. This ensures packages are available immediately and persist across all executions, avoiding any runtime installation cost.

By selecting the correct installation strategy, you can strike a balance between flexibility and faster execution times.

Add your own packages

To define additional dependencies, create a requirements.txt file and place it in the root of your project directory.

Note
 If your project is Git-based, the requirements.txt file must be located in the Code/ folder within your project’s directory.

The requirements.txt file should list all required Python packages using standard pip syntax. You can specify exact versions, minimum versions, or allow the latest versions, along with which libraries to use. For example:

pandas
lxml==3.2.3
numpy>=1.7.1

Domino will automatically install the listed packages at the start of each execution, unless package persistence or preloaded environments are used to optimize startup time.

Install packages interactively in Workspaces

If you’re working in a Jupyter Notebook, you can also install packages interactively using pip directly in a notebook cell:

! pip install --user <package>

The '!' prefix runs the command in the shell from within the notebook environment.

Generate a requirements.txt file

If you’re using pip on your local machine, the easiest way to generate the requirements.txt file is to run the following command in the root of your project folder (or in the Code folder if your project is Git-based):

~/domino/myProject $ pip freeze > requirements.txt

For performance reasons, prune the file to include only the libraries needed for your analysis.

Install packages hosted from a Git repository

Caution
 This is an advanced topic.

Pip can install Python packages from source by cloning a public Git repository over HTTPs. See pip install for reference. To specify this, you must add something like the following line to your requirements.txt file:

-e git+https://git.yourproject.org/you/Project.git#egg=YourProject

The most common host of Git projects is GitHub. If the package you want to install is publicly accessible, then the previous instructions will work. However, if you must install private repositories, Domino can securely integrate with GitHub to access them. See Import Git Repositories for information about securely storing your Github credentials.

Caution
 Do not embed your GitHub credentials directly in the requirements.txt file. Instead, integrate Domino with GitHub securely by following the instructions in Import Git Repositories.