About Domino
Domino Data LabKnowledge BaseData Science BlogTraining

File sync MPI clusters

Note

When using Domino on-demand MPI clusters, data and code from the workspace must be synced with the worker nodes before submitting an mpirun run.

  1. Click the File Changes tab on the navigation bar.

    Note

  2. Click Sync.

    mpi-file-sync.png

    The MPI file sync applies to the following directories:

    • /mnt and subdirectories including /imported for Domino file system projects.

    • /mnt/code, /mnt/artifacts, /mnt/imported and subdirectories for Git-based projects.

    • /repos.

Warning

Domino datasets

When you create an MPI cluster attached to a Domino workspace or job, any Domino dataset accessible from the workspace or job is accessible from all components of the cluster under the same dataset mount path. Worker nodes can also write to attached read/write Domino datasets.

AWS credential propagation

When AWS credential propagation is enabled, temporary AWS credentials are automatically available on all MPI workers and your session. These credentials correspond to the roles enabled in your company’s identity provider.

The credentials are automatically refreshed and available with a profile name corresponding to each role in the AWS credential file. The AWS_SHARED_CREDENTIALS_FILE environment variable contains the location of the file. This variable is in the proper search path for s3fs and boto3. See Connect to generic S3 for more info.