File sync MPI clusters

Note
 The Open MPI feature is only available with Domino versions 5.1.1 and later. Previous versions of Domino do not support this feature.

When using Domino on-demand MPI clusters, data and code from the workspace must be synced with the worker nodes before submitting an mpirun run.

  1. Click the File Changes tab on the navigation bar.

    Note
    		 The tab displays changes since the last sync operation. The mpirun command does not detect changes since the last sync. You must manually sync to use updated code on the cluster worker nodes. Do not execute mpirun commands while files are syncing.

  2. Click Sync under MPI Cluster File Sync.

    The MPI file sync applies to the following directories:

    • /mnt and subdirectories including /imported for Domino file system projects.

    • /mnt/code, /mnt/artifacts, /mnt/imported and subdirectories for Git-based projects.

    • /repos.

Warning
 On-demand clusters in Domino are temporary. Any data stored on cluster local storage, and not externally, is lost when the workload and the cluster shut down.

Domino datasets

When you create an MPI cluster attached to a Domino workspace or job, any Domino dataset accessible from the workspace or job is accessible from all components of the cluster under the same dataset mount path. Worker nodes can also write to attached read/write Domino datasets.

AWS credential propagation

When AWS credential propagation is enabled, temporary AWS credentials are automatically available on all MPI workers and your session. These credentials correspond to the roles enabled in your company’s identity provider.

The credentials are automatically refreshed and available with a profile name corresponding to each role in the AWS credential file. The AWS_SHARED_CREDENTIALS_FILE environment variable contains the location of the file. This variable is in the proper search path for s3fs and boto3. See Connect to generic S3 for more info.