About Domino
Domino Data LabKnowledge BaseData Science BlogTraining

Install configuration reference

KeyDescriptionRequiredValues

schema

YAML schema version.

✓

1.1

name

Unique deployment name. This must contain the name of the deployment owner.

✓

[a-zA-Z0-9_-]+

version

Domino version to install.

✓

Supported versions: 5.3.0

hostname

Hostname Domino application will be accessed at.

✓

Valid fully qualified domain name (FQDN)

pod_cidr

If network policies are enabled, allow access from this CIDR. This range must cover addresses used by your cluster nodes and pods.

Valid CIDR range, for example, 10.0.0.0/8

ssl_enabled

Should Domino only be accessible with HTTPS.

✓

true, false

ssl_redirect

Should Domino only be accessible with HTTPS.

✓

true, false

request_resources

Create Kubernetes resource requests and limits for services.

✓

true, false

enable_network_policies

Use network policies for fine-grained service access.

✓

true, false

Note
 Requires a compatible CNI plugin for example, Calico

enable_pod_security_policies

Enables pod security policies for locked down system capabilities.

✓

true, false

create_restricted_pod_security_policy

Creates pod security policies for locked down system capabilities.

✓

true, false

kubernetes_distribution

Determines resource compatibility with either OpenShift or CNCF Kubernetes

✓

cncf or openshift

external_docker_registry

A URL to ECR (Amazon Elastic Container Registry), ACR (Azure Container Registry), GCR (Google Container Registry) or GAR (Google Artifact Registry). The node must have permissions to the registry.

Note
 Either external_docker_registry or internal_docker_registry must be configured.

custom_certificates

Custom Certificates

✓

Istio

This section configures how and if an Istio Service Mesh is deployed by or integrated to Domino. A Domino-deployed Istio is for Domino use only. These configurations must only be installed and/or enabled if intra-cluster encryption in transit is required.

KeyDescriptionRequiredValues

istio.enabled

Enable Istio in deployment (that is, sidecar injection).

✓

true, false

istio.install

Install Istio service with Domino.

✓

true, false

istio.cni

Configures whether Istio installation is done with a CNI. If true, the installation is done with a CNI and requires fewer permissions; this is our preferred and recommended setting. If false, the installation will add required capabilities to every pod security policy: NET_ADMIN, and NET_RAW.

✓

true, false

Ingress controller

This section configures the NGINX ingress controller deployed by the fleetcommand-agent.

KeyDescriptionRequiredValues

ingress_controller.install

Install the NGINX ingress controller.

✓

true, false

ingress_controller.gcp_static_ip_name

On Google Cloud Platform (GCP), use a static IP address to expose nginx.

The name of the IP address

ingress_controller.class_name

Name of the ingress class for Domino.

✓

nginx

istio_annotations

Include annotations related to Istio.

✓

true,false

Namespaces

Use Namespaces to virtually segment Kubernetes executions. Domino creates namespaces according to the specifications in this section. The installer requires that these namespaces do not exist at installation.

KeyDescriptionRequiredValues

namespaces.platform.name

Namespace to place Domino services.

✓

Kubernetes Names

namespaces.compute.name

Namespace for user executions.

✓

Kubernetes Names

Note
 Might be the same as the platform namespace.

namespaces.system.name

Namespace for deployment metadata.

✓

Kubernetes Names

namespaces.istio.name

Namespace for Istio.

✓

Kubernetes Names

namespaces.*.annotations

Optional annotations to apply to each namespace

Kubernetes Annotation

namespaces.*.labels

Optional labels to apply to each namespace

Kubernetes Labels and Selectors

Storage classes

Storage Classes are a way to abstract the dynamic provisioning of volumes in Kubernetes.

Domino requires the following storage classes:

  1. block storage for Domino services and user executions that need fast I/O.

  2. shared storage that can be shared between multiple executions.

Domino supports pre-created storage classes, although the installer can create a shared storage class backed by NFS or a cloud NFS analog as long as the cluster can access the NFS system for read and write, and the installer can create several types of block storage classes backed cloud block storage systems like Amazon EBS.

Block

KeyDescriptionRequiredValues

storage_classes.block.create

Whether to create the block storage class.

✓

true, false

storage_classes.block.name

The block storage class name.

✓

Kubernetes Name

Note
 Always required due to platform limitations. Cannot be "" which indicates the default storage class.

storage_classes.block.type

Type of the block storage class to use.

✓

  • ebs

  • hostpath

  • gce

  • azure-disk

storage_classes.block.base_path

Base path to use on nodes with hostpath volumes.

storage_classes.block.default

Whether to set this storage class as the default.

✓

true, false

Shared

KeyDescriptionRequiredValues

storage_classes.shared.create

Whether to create the shared storage class.

✓

true, false

storage_classes.shared.name

The shared storage class name.

✓

Kubernetes Name

storage_classes.shared.type

Type of the shared storage class to use.

✓

  • efs

  • nfs

  • azure-file Azure File requires outbound port 445 to be open from your Azure cluster.

storage_classes.shared.efs.region

EFS store AWS region.

For example, us-west-2

storage_classes.shared.efs.filesystem_id

EFS filesystem ID.

For example, fs-7a535bd1

storage_classes.shared.efs.access_point_id

Access Point ID

For example, fsap-01234567

storage_classes.shared.nfs.server

NFS server IP or hostname.

storage_classes.shared.nfs.mount_path

Base path to use on the server when you create shared storage volumes

storage_classes.shared.nfs.mount_options

YAML List of additional NFS mount options.

For example, - mfsymlinks

storage_classes.shared.azure_file.storage_account

Azure storage account to create file stores.

Blob storage

Domino can store long-term, unstructured data in blob storage buckets. Only the shared storage class described previously (NFS and S3) are supported for projects and logs. backups also supports Azure and Google Cloud Storage (GCS) storage.

S3

KeyDescriptionRequiredValues

blob_storage.*.s3.region

AWS region of the S3 bucket store.

For example, us-west-2

blob_storage.*.s3.bucket

S3 bucket name.

For example, domino-bucket-1

blob_storage.*.s3.sse_kms_key_id

KMS Key ID.

For example, 00826efc-2e4b-573d-8d0e-a4ce90811865

Azure

KeyDescriptionRequiredValues

blob_storage.backups.azure.account_name

Azure storage account name.

For example, mystorage

blob_storage.backups.azure.account_key

Access key for the storage account.

For example, domino-bucket-1

blob_storage.backups.azure.container

Name of the container in the storage account.

For example, backups

GCS (Google Cloud Storage)

KeyDescriptionRequiredValues

blob_storage.backups.gcs.bucket

The bucket name.

For example, domino-bucket-1

blob_storage.backups.gcs.service_account_name

The service account name with write access to the bucket.

For example, bucket-sa

blob_storage.backups.gcs.project_name

The service account’s project name.

For example, myproject

Autoscaler

For Kubernetes clusters without native cluster scaling in response to new user executions, Domino supports the use of the cluster autoscaler.

KeyDescriptionRequiredValues

autoscaler.cloud_provider

Cloud provider Domino is deployed with.

  • aws

autoscaler.aws.region

AWS region Domino is deployed into.

For example, us-west-2

autoscaler.azure.resource_group

Azure resource group Domino is deployed into.

Azure resource group

autoscaler.azure.subscription_id

Azure subscription ID Domino is deployed with.

Azure subscription ID

AWS Auto Discovery

The cluster autoscaler supports Auto Discovery on AWS. Without any explicit configuration of specific autoscaling groups, it detects all ASGs that have the appropriate tags and refreshes them if their settings are updated directly. All ASGs must be listed with accurate min/max settings (or not listed at all) is not required as referenced below in the Groups section. ASG settings can be updated directly in AWS. The cluster-autoscaler configuration doesn’t need to be updated, and you don’t need to rerun the installer.

KeyDescriptionRequiredValues

autoscaler.auto_discovery.cluster_name

K8s Cluster Name.

exactly match the name in AWS

autoscaler.auto_discovery.tags

Optional: If filled in, cluster_name is ignored.

For example, - my.tag or []

autoscaler.groups

Must be set to [] if using auto_discovery

By default, if no autoscaler.groups and autoscaler.auto_discovery.tags are specified, the cluster_name will be used to look for the following AWS tags:

  • k8s.io/cluster-autoscaler/enabled

  • k8s.io/cluster-autoscaler/{{ cluster_name }}

The tags setting can be used to explicitly specify which resource tags the autoscaler service must look for.

To disable auto-discovery and use specific groups, ensure that auto_discovery.cluster_name is an empty value.

Groups

Autoscaling groups are not dynamically discovered. Each autoscaling group must be individually specified, along with the minimum and maximum scaling size.

KeyDescriptionRequiredValues

autoscaler.groups.*.name

Autoscaling group name.

Must exactly match the name in the cloud provider

autoscaler.groups.*.min_size

Minimum scaling size.

For example, 0

autoscaler.groups.*.max_size

Maximum scaling size.

For example, 10

External DNS

Domino can automatically configure your cloud DNS provider. See external-dns for more information.

KeyDescriptionRequiredValues

external_dns.provider

Cloud DNS provider.

For example, aws

external_dns.domain_filters

Only allow access to domains that match this filter.

For example, my-domain.example.com

external_dns.zone_id_filters

Only allow updates to specific hosted zones.

txt_owner_id

The owner ID in the TXT record.

Email notifications

Domino supports SMTP to send email notifications in response to user actions and run results.

KeyDescriptionRequiredValues

email_notifications.enabled

Whether Domino must send email notifications.

✓

true, false

email_notifications.server

SMTP server hostname or IP.

email_notifications.port

SMTP server port.

email_notifications.enable_ssl

Whether the SMTP server uses SSL encryption.

✓

true, false

email_notifications.from_address

Email address to send emails from Domino with.

For example, domino @example.com

email_notifications.authentication.username

If you use SMTP authentication, the username.

email_notifications.authentication.password

If you use SMTP authentication, the password.

Monitoring

Domino supports in-cluster monitoring with Prometheus as well as more detailed, external monitoring through New Relic APM and Infrastructure.

KeyDescriptionRequiredValues

monitoring.prometheus_metrics

Install Prometheus monitoring.

✓

true, false

monitoring.newrelic.apm

Enable New Relic APM.

✓

true, false

monitoring.newrelic.infrastructure

Enable New Relic Infrastructure.

✓

true, false

monitoring.newrelic.license_key

New Relic account license key.

Helm

Configuration for the Helm repository that stores Domino’s charts.

KeyDescriptionRequiredValues

helm.version

Which version of Helm to use.

✓

2 or 3

helm.host

Hostname of the chart repository.

✓

For Helm 2 this must be quay.io or the address of your private app server. For Helm 3 it must be gcr.io.

helm.namespace

Namespace to find charts in the repository.

Helm repo namespace. When you use official Domino repositories this must be domino. For Helm 3 with gcr.io or mirrors.domino.tech, use domino-eng-service-artifacts.

helm.username

Username for chart repository if authentication is required. When you use Helm 3 with charts hosted in GCR this must be _json_key.

Username

helm.password

Password for chart repository if authentication is required.

For Helm 3 this is the base64 encoded JSON key that was provided by Domino.

Image registries

List of Docker registries for Domino components.

KeyDescriptionRequiredValues

helm.image_registries.*.server

Docker registry host.

✓

  • quay.io

  • mirrors.domino.tech

helm.image_registries.*.username

Docker registry username.

✓

helm.image_registries.*.password

Docker registry password.

✓

Internal Docker registry

The recommended configuration for the internal Docker registry deployed with Domino. Use override values to allow the registry to use S3, GCS, or Azure blob store as a backend store. GCS requires a service account already be bound into the Kubernetes cluster with configuration to ensure the docker-registry service account is properly mapped.

Note
 Either internal_docker_registry or external_docker_registry must be configured.
KeyDescriptionRequiredValues

internal_docker_registry.s3_override.region

AWS region of the S3 bucket store.

For example, us-west-2

internal_docker_registry.s3_override.bucket

S3 bucket name.

For example, domino-bucket-1

internal_docker_registry.s3_override.sse_kms_key_id

KMS Key ID.

For example, 00826efc-2e4b-573d-8d0e-a4ce90811865

internal_docker_registry.gcs_override.bucket

GCS bucket name.

For example, domino-bucket-1

internal_docker_registry.gcs_override.service_account_name

GCS service account with access to the bucket.

internal_docker_registry.gcs_override.project_name

GCP project name that Domino is deployed into.

internal_docker_registry.azure_blobs_override.account_name

Azure blobstore account name.

internal_docker_registry.azure_blobs_override.account_key

Azure blobstore account key.

internal_docker_registry.azure_blobs_override.container

Azure blobstore container name.

Telemetry

Domino supports user telemetry data to help improve the product.

KeyDescriptionRequiredValues

intercom.enabled

Enable Intercom onboarding.

✓

true, false

mixpanel.enabled

Enable MixPanel.

✓

true, false

mixpanel.token

MixPanel API token.

✓

GPU

If using GPU compute nodes, enable the following configuration setting to install the required components.

KeyDescriptionRequiredValues

gpu.enabled

Enable GPU support.

✓

true, false

Fleetcommand

Domino supports minor patch upgrades through an internal tool named Fleetcommand. To learn more about the telemetry being sent back to Domino, see Deployments Telemetry.

KeyDescriptionRequiredValues

fleetcommand.enabled

Enable ability for Domino staff to apply minor patches.

✓

true, false

fleetcommand.url

The URL to fleetcommand, Domino staff will provide this.

fleetcommand.api_token

Deployment-specific API token (Domino staff will provide this).

Node selectors

Domino will by default deploy some DaemonSets on all available nodes in the host cluster. When you run Domino in a multi-tenant Kubernetes cluster, where some nodes must not be used by Domino, you can label nodes for Domino with a single, consistent label. Then, provide that label to fleetcommand-agent with the below configuration to apply a selector to all Domino resources for that label.

KeyDescriptionRequiredValues

global_node_selectors

List of key/value pairs to use as the label for the selector.

Optional

Example

global_node_selectors:
  domino-owned: "true"

This example applies a selector for domino-owned=true to all Domino deployment resources.

Pod configurations

Global pod configuration that applies to all pods which Domino deploys.

KeyDescriptionRequiredValues

global_pod_annotations

List of key and value pairs to use as annotations that apply to all pods.

Optional

Example

global_pod_annotation:
  my-annotation: "abc"

This example adds an annotation my-annotation to all Domino pod resources.

global_pod_labels

List of key and value pairs to use as labels that apply to all pods.

Optional

Example

global_pod_labels:
  my-label: "abc"

This example adds a label my-label to all Domino pod resources.

global_pod_environment

List of name and value pairs to use as environment variables that apply to all pods.

Optional

Example

global_pod_environment:
  - name: MY_ENV
    value: VALUE

This example adds an environment variable MY_ENV to all Domino pod resources.

Image caching

These settings control the Domino image caching service, which runs as a privileged pod and uses the host Docker socket to pre-pull popular Domino environment images onto compute workers. It can be disabled.

KeyDescriptionRequiredValues

image_caching.enabled

Whether or not to deploy the image caching service.

✓

true, false

Certificate management

KeyDescriptionRequiredValues

certificate_management.install

Whether to install cert-manager. Domino requires cert-manager, but only one instance of cert-manager can be installed in any given Kubernetes cluster. If your Kubernetes cluster already has cert-manager installed, this should be false.

✓

true, false

Note

If cert-manager is already installed, you must add the following configuration override for image building (hephaestus):

release_overrides:
  hephaestus:
    chart_overrides:
      certManagerNamespace: <cert-manager namespace>

Teleport Kubernetes agent

No teleport support is installed if teleport_kube_agent is not present.

KeyDescriptionRequiredValues

teleport_kube_agent.proxyAddr

The teleport address.

✓

teleport_kube_agent.authToken

The authentication token for Teleport.

✓

Elasticsearch

KeyDescriptionRequiredValues

elasticsearch.enable_sysctl

To disable the sysctlInitContainer (which runs as root), set the sysctl vm.max_map_count correctly through some other method.

✓

true, false

Image building

A builder will use a storage cache as large as the amount specified by the cache_storage_size. When a build completes, the storage cache is reduced to the cache_storage_retention amount. The cache_storage_retention value must not be greater than the cache_storage_size.

When the underlying OS does not support user namespace mapping, like EKS, you might have to disable rootless building for deployment targets. Running rootless in an environment that does not support it defaults to using the native filesystem snapshotter and causes image building performance to drop significantly.

KeyDescriptionRequiredValues

image_building.verbose

Enable verbose logging.

✓

true, false

image_building.rootless

Run as a non-root user.

✓

true, false

image_building.concurrency

Maximum number of concurrent image builds.

image_building.cache_storage_size

Size of each image builder’s cache.

image_building.cache_storage_retention

Amount of storage to keep during garbage collection.

image_building.builder_node_selector

List of key/value pairs to use as the label for the selector.

image_building.pool_sync_wait_time

Duration after which builder cluster is inspected for idle pods.

Examples: 1h, 5m, 30s

image_building.pool_max_idle_time

Duration after which idle build pods are terminated.

Examples: 1h, 5m, 30s

image_building.fetch_and_extract_timeout

Duration the build waits to fetch and extract the remote Docker context.

Examples: 1h, 5m, 30s

image_building.pool_endpoint_watch_timeout

Number of seconds the worker pool waits for a buildkit pod to become ready for traffic.

Registries

Configures buildkit/hephaestus to support HTTP-only and/or self-signed registries.

KeyDescriptionRequiredValues

image_building.registries.*.http

Registry only supports HTTP.

✓

true, false

image_building.registries.*.insecure

Registry uses a self-signed certificate and is considedred insecure.

✓

true, false

Cloud registry authentication

Set up cloud authentication so that the image builder can pull images from and push images to cloud container registries.

Azure

KeyDescriptionRequiredValues

image_building.cloud_registry_auth.azure.tenant_id

Azure AD Directory (tenant) ID.

✓

image_building.cloud_registry_auth.azure.client_id

Azure AD Application (client) ID.

✓

image_building.cloud_registry_auth.azure.client_secret

Azure AD Application secret, required when using a Service Principal.

Google Cloud Platform (GCP)

KeyDescriptionRequired

image_building.cloud_registry_auth.gcp.service_account

Service account with access to target GCR/GAR registry.

✓