About Domino
Domino Data LabKnowledge BaseData Science BlogTraining

Set up drift detection

To monitor a model’s data drift you must ingest training data. Training data is the data used while constructing a model that leverages machine learning techniques. Training data records must include both model input values and a previously-known corresponding output for each input value combination from which the model can learn.

Your model’s training data is used to analyze data drift. This topic describes how to register a training set for monitoring purposes.

Prerequisites

Before you proceed, you must create a training set that was used to build a model (.pkl file), and then that model was deployed into a model API.

Select the training set

  1. On the model API’s page, click Monitoring. The page shows the Monitoring is waiting for data message until you configure training data and ground truth data.

  2. Click Configure Monitoring > Data.

    The Monitoring page shows the Waiting for data message.

  3. From the Configure Data window, select the Training Data and Version.

  4. Select Classification or Regression, depending on your model type. If your Training Set code includes prediction data defined in target_columns, select the model type that matches your Training Set:

    • If target_columns is a categorical column, select Classification.

    • If target_columns is a numerical column, select Regression.

      Complete the fields in the Configure Data window.

  5. Click Save.

Note

Validate your Setup

Domino captures prediction data in a Domino Dataset in hourly batches.

Validate your setup without waiting for the Scheduled Check to run:

  1. On the Model API’s Monitoring tab, go to Configure Monitoring > Target Ranges.

    Use the Configure Tests and Thresholds page to experiment with test and threshold settings.

  2. If the Configure Tests and Thresholds page populates with data, then your setup is complete. See Analyze Data Drift to learn how to configure tests and thresholds.