Domino’s integration with NVIDIA Fleet Command simplifies model deployment to the edge.
Scaling AI at the edge is a critical way of deploying data science work, and is useful for scenarios like anomaly detection at cellphone towers and quality control in manufacturing.
-
Deploy a model API from any project and run tests to ensure your model API is functioning properly.
-
After the deployment status changes to Ready to Run, invoke the Domino API call to register the model image with NVIDIA.
/:modelId/:modelVersionId/exportImageForNvidia -
Wait for the API to register the model API container image and an associated Helm chart to the Fleet Command registry.
See the model API details to learn how to invoke the export API to push the image to Fleet Command and check the status endpoint to track the progress of the export.
-
Use Fleet Command to configure the distributed edge system.
With Domino’s model export catalog, you can see a unified view of all exported models and their performance. With this view, you can track and manage all production assets from a single pane.
curl =
--location --request POST 'https://<deployment>/v4/models/<model-id>/<model-version-id>/exportImageForNvidia'
--header 'Content-Type: application/json'
--header 'X-Domino-Api-Key: <domino-api-key>'
--data-raw '{
"containerRepository": "<path-to-container-repo>",
"tag": "<image-tag>",
"helmRepository": "<path-to-helm-repo>",
"helmVersion": "<version>",
"ngcApiKey": "<ngc-api-key>"
}'Here is a description of the parameters:
-
model-id: unique identifier for the Model API -
Model-version-id: version ID for the Model API -
containerRepository: Repository name for a private container repository in NVIDIA NGC. For example, <organization-name>/<team-name>/<container-repository-name>. The user can provide a container repository that already exists in NVIDIA NGC. Here are the acceptable elements for a Container repository name:-
Alphanumeric characters
-
Lowercase characters
-
Dashes and slashes (for organization and team names)
-
No spaces
-
-
tag: Tag for the container. The tag must be alphanumeric. It can also contain underscores, periods, and dashes. -
helmRepository: Repository name for a private helm repository in NVIDIA NGC. For example, <organization-name>/<team-name>/<container-repository-name>. The user can provide a helm repository that already exists in NVIDIA NGC. Here are the acceptable elements in a Helm repository name:-
Alphanumeric characters
-
Lowercase characters
-
Dashes and slashes (for organization and team names)
-
No spaces
-
-
helmVersion: Version of the Helm chart. Helm versions must be in Semver format. -
ngcApiKey: NVIDIA NGC API key. See generate an NVIDIA NGC API Key.Note-
The port for the endpoint on the edge device is hardcoded to 30008. If you deploy two Domino models on the same edge device, the second application deployed fails. Override the port in
values.yamlin the NVIDIA Fleet Command UI to prevent this failure. -
The model cannot use anything Domino-specific like predictive sets or domino datasets.
-
The model must be built in Domino first, and there must be a version of the Model.
-
If you export a model with an image tag or Helm chart version number that already exists, the export fails.
-
You cannot deploy different versions of the same NVIDIA Application to the same edge device. If you want to deploy multiple Domino models on a single-edge device, override the port when you deploy.
-
Use the export-id you get as the return value from the export endpoint to monitor the export status.
Here is a sample command:
curl --include
--header 'Content-Type: application/json'
--header 'Accept: application/json'
--header 'X-Domino-Api-Key: <domino-api-key>'
'https://<deployment>/v4/models/<export-id>/getExportImageStatus'