Configure Dask prerequisites

Before you can start using on-demand Dask clusters on Domino, you must ensure that this functionality is enabled and properly configured on your deployment.

Note

Domino on-demand Dask functionality is available starting with Domino 4.6.

Create a base Dask cluster environment

When using on-demand Dask in Domino you will have two separate environments: one for the Dask cluster (base or worker environment) and one for the workspace/job execution (compute environment).

You can get the latest released Dask images from compute Environment catalog.

To create a new base Dask cluster environment, follow the general Environment management instructions with the following environment_attributes:

  • Base Image

    Select Custom Image and enter an image URI that points to a deployable Dask image.

    From the options published at compute Environment catalog, select the Dask cluster image available in the cluster-environment-images repository. Domino’s repository contains the latest Dask images curated for Domino.

    The available images include the full set of Dask components and common dependencies like Pandas and NumPy.

  • Supported Clusters

    Select the Domino managed Dask option (required). This ensures that the environment will be available for use when creating Dask clusters from workspaces and jobs.

  • Visibility

    Set this attribute the same way you would for any other compute environment.

  • Dockerfile Instructions

    Leave blank to use the base image provided by the Dask community.

    You can modify this section to include additional packages that might be necessary for your workloads and must be available on the Dask cluster nodes.

    See Manage dependencies to learn more.

  • Pluggable Notebooks / Workspace Sessions

    Leave this section blank as the Dask base environments are not intended to include notebook configuration.

Prepare your Dask execution compute environment

In addition to the base Dask cluster environment, you must also configure the Dask compute environments for workspaces and/or jobs that will connect to your cluster.

Domino recommends that you use the Dask base image to create a compatible workspace. See compute Environment catalog.

Customize this Workspace compute environment:

Use the image mentioned previously and add Pluggable Workspace Tools.

jupyter:
  title: "Jupyter (Python, R, Julia)"
  iconUrl: "/assets/images/workspace-logos/Jupyter.svg"
  start: [ "/opt/domino/workspaces/jupyter/start" ]
  supportedFileExtensions: [ ".ipynb" ]
  httpProxy:
    port: 8888
    rewrite: false
    internalPath: "/{{ownerUsername}}/{{projectName}}/{{sessionPathComponent}}/{{runId}}/{{#if pathToOpen}}tree/{{pathToOpen}}{{/if}}"
    requireSubdomain: false
jupyterlab:
  title: "JupyterLab"
  iconUrl: "/assets/images/workspace-logos/jupyterlab.svg"
  start: [  "/opt/domino/workspaces/jupyterlab/start" ]
  httpProxy:
    internalPath: "/{{ownerUsername}}/{{projectName}}/{{sessionPathComponent}}/{{runId}}/{{#if pathToOpen}}tree/{{pathToOpen}}{{/if}}"
    port: 8888
    rewrite: false
    requireSubdomain: false
vscode:
  title: "vscode"
  iconUrl: "/assets/images/workspace-logos/vscode.svg"
  start: [ "/opt/domino/workspaces/vscode/start" ]
  httpProxy:
    port: 8888
    requireSubdomain: false
rstudio:
  title: "RStudio"
  iconUrl: "/assets/images/workspace-logos/Rstudio.svg"
  start: [ "/opt/domino/workspaces/rstudio/start" ]
  httpProxy:
    port: 8888
    requireSubdomain: false