Domino 5.1.3 (May 2022)

This release provides security patches and improvements.

See also the fleetcommand-agent Release Notes.

Improvements

Compatibility with Kubernetes 1.21.

Handle invalid environment revisions

In Domino 5.0.0 through 5.1.2, Mongo records can have a Docker image or a base environment revision but not both. When upgrading from Domino 4.x to 5.1.3 and above, Domino uses the base environment revision to gracefully handle environment revisions that have both a Docker image and a base environment revision.

Note
When upgrading from Domino 4.x to 5.1, contact Domino’s Customer Success team for access to the migration script that provides a workaround for this issue.

Resolve Vault and Kubernetes 1.21 issue

In prior versions of Domino, there were Vault failure issues with Kubernetes 1.2.1. This issue is resolved in the 5.1.3 release.

Note	This release includes new environment-specific installation steps for GCP deployments.

Known issues

Workspaces become unstoppable and their statuses don’t change when the timestamps on the run’s status changes array are incorrect.

After deleting branches that were created in GitHub but not pulled (that is, they are not local on Domino), they are still shown in the Code menu in the Data tab of the workspace.

If you are comparing jobs that have large numbers of output files, you might encounter a 502 Bad Gateway error.

In the Control Center’s navigation pane, click Assets. Domino displays a "Failed to get assets" message and takes a long time to load the assets.

You cannot delete, rename, or move a .gitignore file in a Domino File System project.
- If you delete the file, it remains.
- If you move the file, the original and target file both exist.
- If you rename the file, the source and renamed file both exist.

If you rerun a successful job that accesses files from a dataset in a project owned by an organization, the job fails with file not found errors. Initial runs of the jobs succeed, but reruns fail with logging similar to the following:
```
2022-10-06 04:34:32 : FileNotFoundError: [Errno 2] No such file or directory: '/domino/datasets/local/reruns_fail/file.txt'
```
To work around this, start the execution as a new job.

If you use a forward slash (/) in a Git branch name, the forward slash and any preceding text is stripped. For example, if the branch was named test/branch1, it will be renamed branch1.
- In a workspace in a Git-based project, you can’t pull data because the branch doesn’t exist. You can push because this creates a new branch named branch1.
- In a Domino File System project that uses imported code repositories, you can push and pull directly to and from the branch (test/branch2) if you don’t change the branch in the menu. If you change the branch, you can’t return to the selection (test/branch2) because test/ is stripped. If you push to branch2, this creates a new branch named branch2.

The graph views are skewed incorrectly, in Domino’s model monitoring, when normalized values are small.
When exporting notebooks from JupyterLab workspaces, the system shows a 403 Forbidden Error with Blocking Cross Origin message.
The output of the gateway/runs/getByBatchId endpoint is missing the runs and nextBatchId objects. This issue is resolved in Domino 5.2.0.

When viewing a raw HTML file while com.cerebro.domino.frontend.allowHtmlFilesToRender is false, Javascript can be executed. This issue is fixed in Domino 5.2.0.

When you set an environment as the default on the Environments page and then restart the service, Domino reverts to the default environment previously set on the Central Configuration page. This issue is fixed in Domino 5.2.0.

Sync operations for deployments backed by SMB file stores can sometimes result in unintended file deletions. This issue is fixed in Domino 5.2.0.

You cannot delete a .gitignore file in a Domino File System (DFS)-based project. This issue is fixed in Domino 5.2.0.

Jobs can fail or become stuck in a Pending state when using files from an S3 data source. This issue is fixed in Domino 5.2.0.

Entering a URI in the New Environment window can cause the window to go blank. This issue is resolved in Domino 5.2.0.

Apps may fail with unusual error messages. This issue is resolved in Domino 5.2.0.

The Domino UI may hang with a "Waiting for available socket" error. This issue is resolved in Domino 5.2.0.

When an API key explicitly passed as an api_key argument and the DOMINO_TOKEN_FILE environment variable is set, Domino uses the token authentication. This issue is resolved in Domino 5.2.0.

When Flask apps generate links, this can result in unwanted 302 redirect behavior. This issue is fixed in Domino 5.2.0.

When a Model API fails while retrieving files from git using SSH keys, the private key and the key password, if any, may be exposed in the Domino log. This issue is resolved in Domino 5.2.0.

The Import Project drop-down does not display projects owned by organizations of which you are a member. This issue is resolved in Domino 5.2.0.

MATLAB environment builds may fail, or MATLAB workspaces may be unresponsive.

You might experience slow loading times when you open the Environments, Workspaces, Model APIs, or Projects pages.

When starting a workspace, the application sometimes gets stuck in the Pulling status, even though the logs indicate that the workspace started successfully. If this occurs, stop and start the workspace again. This issue is fixed in Domino 5.2.0.

Setting com.cerebro.domino.workloadNotifications.isRequired to true does not prevent users from turning off notifications, and changing the value of com.cerebro.domino.workloadNotifications.maximumPeriodInSeconds has no effect.

When you add environment variables to a project, they are visible in the execution_specifications collection in MongoDB.

After upgrading to 4.6.4 or higher, launching a job from a Git-based project that uses a shared SSH key or personal access token results in a Cannot access Git repository error.

In a Git-based project, requirements.txt must be in your project’s Artifacts folder in order to be picked up by an execution, instead of at the root level in your project files.

Tip
To avoid falling out of sync with the Git repository, always modify requirements.txt in the repository first and then copy it to your project’s Artifacts folder.

After you migrate from 4.x, scheduled jobs will fail until the users that own them log out and log in again. This issue is resolved in Domino 5.3.0.

Tip
You can also resolve this issue by removing all existing tokens in MongoDB, like this: db.user_session_values.remove({"key": "jwt"})

Domino can become inaccessible after a fixed uptime period when running on Kubernetes 1.21 or higher. See this knowledge base article for the workaround. This issue is fixed in Domino 5.3.1.

Upgrading to this release from Domino 3.x or 4.x can cause Git credentials to be omitted from the dropdown when creating a new Git-based project. A workaround is available from Domino’s Customer Success team.

Model Monitoring data sources aren’t validated. If you enter an invalid bucket name and attempt to save, the entry will go through. However, you won’t be able to see metrics for that entry because the name points to an invalid bucket.

A limit overcommit error (OOMKilled) is occurring frequently on the prometheus-adapter pod in Domino’s Integrated Model Monitoring. The current workaround is to increase the limits. This issue is resolved in Domino 5.5.0.

If the Model Access Token Vault migration runs more than once, it wipes tokens from Mongo and Vault, and users need to regenerate their Model Access Tokens in the UI. This is fixed in Domino 5.4.0.

"Other" Git service provider URLs not ending in ".git" fail to work when you attempt to create a project from a Git repo or add a Git repo to a project. This issue is fixed in Domino 5.5.0.

Configuring prediction when setting up prediction capture for Model APIs causes predictions to not be captured. This issue is resolved in 5.4.0.

When restarting a Workspace through the Update Settings modal, External Data Volumes are not mounted in the new Workspace. Follow the steps to mount External Data Volumes. This issue is fixed in Domino 5.9.0.

Viewing dataset files in an Azure-based Domino cluster may lock files, preventing them from being deleted or modified. Restarting Nucleus frontend pods will release the lock. This issue is fixed in Domino 5.11.1.