NVIDIA DGX best practices

Domino recommends you do not use a DGX GPU as a build node for environments. Instead, opt for a CPU resource as part of your overall Domino architecture.

Domino recommends providing several GPU tiers with different numbers of GPUs in each tier. For example, 1, 2, 4, and 8 GPU hardware tiers as different training jobs can take use of single or parallel GPU usage and consuming a whole DGX box for one workload might not be feasible in your environment.

After splitting up hardware tiers, access can be global or, alternatively, limited to specific organizations. Domino recommends that you ensure that the right organizations have GPU Hardware Tier access, or are restricted, to ensure availability for critical work, and/or to prevent the unauthorized use of GPU tiers.