domino logo
Latest (5.8)
  • Overview
  • Domino Cloud
  • Domino Nexus
  • Code Assist
  • Get started
  • Work with data
  • Develop models
  • Scale out distributed computing
  • Register and govern models
  • Deploy models
  • Monitor models
  • Publish Apps
  • Projects
  • Collaborate
  • Workspaces
  • Jobs
  • Environments
  • Executions
  • Launchers
  • Environment variables
  • Secure credential store
  • Organizations
  • Domino API
  • Domino CLI
  • Troubleshooting
  • Get help
  • Additional resources
  • Send feedback
domino logo
About Domino
Domino Data LabKnowledge BaseData Science BlogTraining
>
User guide
>
Scale out distributed computing
>
Distributed GPUs with Open MPI
>
File sync MPI clusters

File sync MPI clusters

Note

When using Domino on-demand MPI clusters, data and code from the workspace must be synced with the worker nodes before submitting an mpirun run.

  1. Click the File Changes tab on the navigation bar.

    Note
  2. Click Sync.

    mpi-file-sync.png

    The MPI file sync applies to the following directories:

    • /mnt and subdirectories including /imported for Domino file system projects.

    • /mnt/code, /mnt/artifacts, /mnt/imported and subdirectories for Git-based projects.

    • /repos.

Warning

Domino datasets

When you create an MPI cluster attached to a Domino workspace or job, any Domino dataset accessible from the workspace or job is accessible from all components of the cluster under the same dataset mount path. Worker nodes can also write to attached read/write Domino datasets.

AWS credential propagation

When AWS credential propagation is enabled, temporary AWS credentials are automatically available on all MPI workers and your session. These credentials correspond to the roles enabled in your company’s identity provider.

The credentials are automatically refreshed and available with a profile name corresponding to each role in the AWS credential file. The AWS_SHARED_CREDENTIALS_FILE environment variable contains the location of the file. This variable is in the proper search path for s3fs and boto3. See Connect to generic S3 for more info.

Domino Data Lab
Knowledge Base
Data Science Blog
Training
Copyright © 2023 Domino Data Lab. All rights reserved.