Export models to NVIDIA Fleet Command

Domino’s integration with NVIDIA Fleet Command simplifies model deployment to the edge.

Scaling AI at the edge is a critical way of deploying data science work, and is useful for scenarios like anomaly detection at cellphone towers and quality control in manufacturing.

Export to NVIDIA Fleet Command

  1. Deploy a Domino endpoint from any project and run tests to ensure your Domino endpoint is functioning properly.

  2. After the deployment status changes to Ready to Run, invoke the Domino API call to register the endpoint image with NVIDIA.

    /:modelId/:modelVersionId/exportImageForNvidia

  3. Wait for the API to register the Domino endpoint container image and an associated Helm chart to the Fleet Command registry.

    See the Domino endpoint details to learn how to invoke the export API to push the image to Fleet Command and check the status endpoint to track the progress of the export.

  4. Use Fleet Command to configure the distributed edge system.

With Domino’s model export catalog, you can see a unified view of all exported models and their performance. With this view, you can track and manage all production assets from a single pane.

Invoke the export API

curl =
--location --request POST 'https://<deployment>/v4/models/<model-id>/<model-version-id>/exportImageForNvidia'
--header 'Content-Type: application/json'
--header 'X-Domino-Api-Key: <domino-api-key>'
--data-raw '{
 "containerRepository": "<path-to-container-repo>",
 "tag": "<image-tag>",
 "helmRepository": "<path-to-helm-repo>",
 "helmVersion": "<version>",
 "ngcApiKey": "<ngc-api-key>"
}'

Here is a description of the parameters:

  • model-id: unique identifier for the Domino endpoint

  • Model-version-id: version ID for the Domino endpoint

  • containerRepository: Repository name for a private container repository in NVIDIA NGC. For example, <organization-name>/<team-name>/<container-repository-name>. The user can provide a container repository that already exists in NVIDIA NGC. Here are the acceptable elements for a Container repository name:

    • Alphanumeric characters

    • Lowercase characters

    • Dashes and slashes (for organization and team names)

    • No spaces

  • tag: Tag for the container. The tag must be alphanumeric. It can also contain underscores, periods, and dashes.

  • helmRepository: Repository name for a private helm repository in NVIDIA NGC. For example, <organization-name>/<team-name>/<container-repository-name>. The user can provide a helm repository that already exists in NVIDIA NGC. Here are the acceptable elements in a Helm repository name:

    • Alphanumeric characters

    • Lowercase characters

    • Dashes and slashes (for organization and team names)

    • No spaces

  • helmVersion: Version of the Helm chart. Helm versions must be in Semver format.

  • ngcApiKey: NVIDIA NGC API key. See generate an NVIDIA NGC API Key.

    Note

    • The port for the endpoint on the edge device is hardcoded to 30008. If you deploy two Domino models on the same edge device, the second application deployed fails. Override the port in values.yaml in the NVIDIA Fleet Command UI to prevent this failure.

    • The model cannot use anything Domino-specific like predictive sets or domino datasets.

    • The model must be built in Domino first, and there must be a version of the Model.

    • If you export a model with an image tag or Helm chart version number that already exists, the export fails.

    • You cannot deploy different versions of the same NVIDIA Application to the same edge device. If you want to deploy multiple Domino models on a single-edge device, override the port when you deploy.

Track the status of the export

Use the export-id you get as the return value from the export endpoint to monitor the export status.

Here is a sample command:

curl --include
    --header 'Content-Type: application/json'
    --header 'Accept: application/json'
    --header 'X-Domino-Api-Key: <domino-api-key>'
 'https://<deployment>/v4/models/<export-id>/getExportImageStatus'

Next steps

Integrate endpoint deployment with CI/CD workflows