Use the Domino API to export Domino model API endpoints to AWS SageMaker inference endpoints.
With a single API call, Domino bundles the model, prediction code, files, packages, and environment into a container image compatible with AWS SageMaker.
After your Domino model API deploys and is up and running, you can send the following POST
call to build a Docker image that is compatible with AWS SageMaker. Domino exports the Docker image to AWS ECR or any third-party container registry.
POST /:modelId/:modelVersionId/exportImageForSagemaker
As part of the API request, you provide credentials for the registry. These credentials are not saved in Domino and optionally work within a time-to-live (TTL) duration.
{
"registryUrl": "ECR_URL_in_your_AWS_account", // ex. 1234567890.dkr.ecr.us-east-2.amazonaws.com
"repository": "my_repo", // name of the repository in ECR
"tag": "tag1", // (Optional)
"username": "AWS", // (Optional)
"password": "xxtokenxx" // (Optional)
}
{
"modelId": "5555dbbd017416e4515555",
"modelVersionId": "3333dbbd017416e4513333",
"exportId": "3333dbbd017416e4513333",
"status": "preparing"
}
Domino updates your model API image to comply with SageMaker inference requirements by doing the following:
-
Adds an
ENTRYPOINT
instruction to the exported Docker image. To execute this in AWS, run it with'docker run image serve'
. -
Adds the
/invocations
endpoint to the container. This endpoint serves the model predictions. -
Adds the
/ping
endpoint to the container. This allows the hosted model to respond to periodicGET
requests that track the state of the endpoint. -
Binds the container’s port to 8080, as required by SageMaker to serve predictions.
Note
| The API only builds and exports the image. It does not deploy it. You must deploy the image to SageMaker as part of your CI/CD or deployment workflow. |
With Domino’s model export catalog, you can see a unified view of all exported models and their performance. With this view, you can track and manage all production assets from a single pane.
When you export a Domino Model to AWS SageMaker and create an endpoint from the exported Model, the endpoint might fail with the error "The primary container for production variant variant-name-1 did not pass the ping health check."
Use these steps to work around the issue:
-
Add
USER root
in the environment Dockerfile instructions. -
Publish the model API from the new instructions.
-
Export the same image in AWS SageMaker and create an endpoint.