domino logo
5.1
  • Get started
  • Tech ecosystem
  • Domino Cloud
  • Develop models
  • Work with data
  • Deploy models, apps, launchers
  • Model Monitoring
  • Code Assist
  • Security and credentials
  • Organizations
  • Collaborate
  • Projects
  • Workspaces
  • Environments
  • Executions
  • Additional Resources
  • Domino CLI
  • Troubleshooting
  • Get help
domino logo
About Domino
Domino Data LabKnowledge BaseData Science BlogTraining
>
User Guide
>
Model Monitoring
>
Data Drift and Quality Monitoring

Data Drift and Quality Monitoring

Domino monitors data drift and model quality.

Data drift metrics compare training data and prediction data

Data drift

Data drift occurs when production data diverges from the model’s original training data. Data drift can happen for many reasons, including a changing business environment, evolving user behavior and interest, modifications to data from third-party data sources, data quality issues, and even issues in upstream data processing pipelines. Data drift monitoring compares live predictions with the model’s training data, and then sends an alert when live predictions diverge too much from the training data. See Analyze Data Drift.

Model quality

Model quality monitoring compares the model’s predicted values against the actual results (or labels for the predictions) using ground truth data to generate quality metrics. For classification models, Domino reports the following metrics:

  1. Accuracy

  2. Precision

  3. Recall

  4. F1

  5. AUC ROC

  6. Log Loss

  7. Gini (Normalized)

Domino also provides a confusion matrix and classification report for further quality evaluation.

For regression models, Domino reports the following metrics.

  1. Mean Square Error (MSE)

  2. Mean Absolute Error (MAE)

  3. Mean Absolute Percentage Error (MAPE)

  4. R-Squared (R2)

  5. Gini (Normalized)

See Metrics for classification and regression models for details.

See Analyze Model Quality.

Remediation

When data drift or model quality issues are detected, the Cohort Analysis feature identifies underperforming cohorts of data so you can take remedial action. See Cohort Analysis.

Use Domino to recreate the development environment originally used to train the model. Then, you can use this environment to access the prediction data to diagnose issues, update code, and retrain your model with the latest production data.

Domino Data Lab
Knowledge Base
Data Science Blog
Training
Copyright © 2022 Domino Data Lab. All rights reserved.