Create Environments from external custom images

This topic describes how to configure Domino so that you can create Environments based on custom images hosted in external, authenticated image registries.

This topic is useful if you want to:

Enable your users to access an internal registry of images to use in Domino.
Enable your users to access an authenticated third-party registry containing images related to your data science workflows. For example, the NVIDIA Container Registry (NGC).

Pre-requisites

Access to the Kubernetes cluster where Domino is running and permissions to read and edit secrets in the Domino compute namespace.
Access to and familiarity with using kubectl. You must be comfortable editing Kubernetes objects.
Access to Basic Authentication (username and password) credentials for the authenticated image registry you want to configure.
The registry must support HTTPS. If it doesn’t, contact support or your field engineer.

Note	The supplied registry credentials will be configured Domino-wide. Therefore, they must be safe for use by all Domino platform users, not just specific users. Domino recommends creating a service account in the image registry specifically designated for use with the Domino platform.

Add authentication details

Use this procedure to add authentication details to your registry secret. Domino’s image builder uses this secret to authenticate with registries during environment builds.

Determine the registry authentication secret name and namespace
1. Edit the secret. The default location for the secret is in the Domino compute namespace, under the secret name, domino-quay-repos. However, another user might have changed the location or secret name.
2. Because the secret name and namespace values can be changed, go to Advanced > Central Config to confirm that you do not have com.cerebro.domino.builder.remoteRegistryCredentials.secretName or com.cerebro.domino.builder.remoteRegistryCredentials.secretNamespace set. If either of these are set use those values instead of the default values listed.
Fetch the current secret value
1. Using the secret name and namespace from the previous steps, run the following command:
  kubectl get -n <secret namespace> secret <secret name> -o jsonpath='{.data}'
  Your output should look similar to the following: {".dockerconfigjson":"…"}
  
  The value of .dockerconfigjson is the value in the double quotes ("") after .dockerconfigjson.
  
  Retain this value as it will be used in subsequent steps.
2. Using the value you retained in the previous step, run the following command:
  echo '<the copied value>' | base64 --decode
3. Copy the output of this command into a text editor of your choice. Ensure you do not copy the final % symbol. The value you copy should look similar to this: {"auths": {"[quay.io]": {"username": "…", "password": "…", "email": "."}}}

Add the new registry authentication

Use your text editor to add your new registry to the previous content. You’ll want to add it at the same level as the existing repositories within the auths dictionary.
The key will be the repository host. For example nvcr.io for NGC. The value of this key is a dictionary with a username and password entry with your username and password.

You can use this template, replacing the values with the pre-existing values and your new values:

{"auths": {"quay.io": {"username": "...", "password": "...", "email": "."}, "<new registry>": {"username": "...", "password": "..."}}}

Note	If you are using an NGC container, you can use this template with the required username: {"auths": {"quay.io": {"username": "...", "password": "...", "email": "."}, "nvcr.io": {"username": "$oauthtoken", "password": "..."}}}

You might want to keep your the text editor open until all the steps are complete. This will save time if you must debug an error later.

Encode the new secret content
1. Copy the new text from your editor. Ensure it is all on one line and does NOT have a newline at the end.
2. Using the text copied from your text editor, run this command:
  echo -n '<your copied content>' | base64 -w 0`.
  You MUST use single quotes to properly escape the double quotes in the copied content.
3. Copy the output of that command. Ensure you do not to copy the trailing % character.
Edit the secret
If you want to edit the secret interactively, run:
```
kubectl edit -n <secret namespace> secret <secret name>
```
1. In the system editor, remove the content after .dockerconfigjson replacing it with the previous encoded text.
2. If you want to edit the secret non-interactively, run:
  kubectl get -n <secret namespace> secret <secret name> -o json | jq '.data[".dockerconfigjson"]="<your copied content>"' | kubectl apply -f -
Validate your edits
1. Repeat the step where you’ve fetched the current secret value before and ensure the decoded content has your updates. Carefully validate the dictionary structure.
Verify your changes
1. Go to Domino and create a new Domino Environment in the Environments tab.
2. Use an image URI from the new authenticated registry as the custom base image for the Environment.
3. Watch the build progress in the Environment revisions tab. Ensure the build succeeds.
4. If the build succeeds, delete all credentials you saved on your system in the text editor.
  
  If the build fails, use the revision build logs to determine if the error is related to authentication. Double check that your credentials are correct and validate that you edited the secret properly, repeating the previous steps as necessary.
  
  If your edits do not work and you can not find any errors, see the previous steps to edit the secret but use the original encoded content that you retained in the step where you fetched the current secret value.