In Domino, you can monitor models deployed on Domino and elsewhere. For Domino models deployed as Domino endpoints, you can use our Domino endpoint monitoring workflow, so you don’t have to maintain access to external data sources. For models deployed in other formats (such as an App, Launcher, or Job) or external to Domino, you can use our Model Monitor workflow.
Use Domino’s Model Monitor to see a single list of monitored models no matter how they are deployed.
When a model is deployed on Domino as a Domino endpoint, Domino:
-
Analyzes the training data to extract the model schema (if you register a Domino TrainingSet).
-
Captures predictions as Domino datasets for monitoring.
-
Generates drift detection and model quality analysis on a schedule (if you share the ground truth dataset with Domino), and alerts you if any thresholds are exceeded.
-
Allows you to easily reproduce the environment with access to the captured predictions to diagnose and fix issues with your model.
If you do not want Domino to manage the prediction data collection, use the Model Monitor to configure monitoring, even for Domino endpoints.
For models deployed as other assets on Domino (App, Launcher, or Job) or external to Domino, you can use Domino to:
-
Connect to the data source where the training, prediction, and ground truth data reside.
-
Register a model’s entry along with its schema.
-
Set up drift detection and model quality monitoring by registering the location of every new batch of prediction or ground truth data.
-
Set up a schedule for Domino to run drift and model quality checks periodically and alert you if thresholds are exceeded.
See Set up Model Monitor.
Contact your Domino support person if you need more help in determining the best workflow.