Sometimes you may want to build models in Domino but host them in an environment outside Domino. This is for reasons such as you may have already made huge investments in the production environment outside Domino that supports very high scale or low latency, the production data can not be exported outside of an environments due to legal/compliance issues, you want to control deployment using a custom CI/CD pipeline, etc.
Domino allows you to export model images built in Domino to an external container registry. These images include all the information needed to run the model including the model code, artifacts, environment, project files, etc.
Domino has exposed REST APIs that can be called by your CI/CD pipeline or workflow to programmatically build and export the model Image. By default the images are built in Model API format. These images can be easily deployed in an environment that can run docker containers. However, you can also export images which are in AWS Sagemaker-compatible format so that they can be directly deployed in AWS Sagemaker. The following APIs are now available:
This API builds a docker image for a model and stores it in Domino’s internal Registry. This can later be fetched from the registry (using other export APIs) by your CI/CD pipeline. The exported Image can be deployed in the your runtime env (outside Domino). Your CI/CD pipeline can add more layers to this image to do further customizations (such as adding Auth, etc).
This API can be used to Push a Model Image to a 3rd party container registry outside Domino. It assumes the Image was already built and available within Domino. As part of the API request, users need to provide credentials for their registry to push Image to it. These credentials are not saved inside Domino and can have a TTL attached.
This API builds a docker image for a given version of an Model API in a AWS Sagemaker-compliant format and then exports it to AWS ECR or any 3rd party container registry outside Domino. As part of the API request, users need to provide credentials for their registry to push Image to it. These credentials are not saved inside Domino and can have a TTL attached.
Sagemaker has the capability to train and then deploy a model to serve requests. Model export functionality in Domino only supports the serve use case since train operation would have already happened in Domino. The Image is ready to be deployed in the Sagemaker environment. All the necessary files required to make predictions are packaged inside the container.
When you export a Domino model to Amazon Sagemaker and create an
endpoint from the exported model, the Sagemaker endpoint may fail with
The primary container for production variant variant-name-1 did not pass the ping health check.
Follow these steps to work around the issue:
USER rootin the environment
Publish the model API from the new instructions.
Export the same image in Sagemaker and create an endpoint.