After reviewing the data drift and model quality monitor, use the monitoring results and Cohort Analysis to determine whether there are concerns with your model. If you do have concerns, review your data.
If you want to review the model and its associated code to investigate further, you can reproduce the original code commits and artifacts in the environment in which you deployed the model. If necessary, you can update the model code or retrain it with the latest production data. Then you can deploy a new, improved version of the Model API.
Domino automatically creates a prediction dataset named prediction_data for every project that can be accessed from any workspace. The predictions are in Parquet format and are updated hourly as the Model API processes inputs. If there is no data in an hour, no file is created. If you configured data drift monitoring or Set up Model Quality Monitoring then the stored prediction data is automatically consumed by the Model Monitor.
By default, a daily job deletes data older than 30 days. Your administrator defines the retention policy for predictions. See the model monitoring configuration options.
Review your data:
-
In your workspace, open the IDE.
-
Use the following paths to read the data:
-
To load individual Parquet files:
-
/domino/datasets/local/prediction_data/<model_version_id>/$$date$$=<date_in_utc>/$$hour$$=<hour_in_utc>/predictions_<uuid>.parquet|
Caution
|
Do not rename the generated Parquet files and directories. Doing so can cause inconsistent behavior. |
Remediate the model
-
From the navigation pane, click Model APIs.
-
Click the model that you want to remediate and then click Open in Workspace.
-
From the Open in New Workspace and Branch window, type a name for the workspace.
-
Select a Hardware Tier.
-
In New Branch Name, type a name for the code branch.
-
Click Open. A Domino workspace opens and is ready for you to take remedial action.
NoteIf you manage file changes and code commits outside of Domino (such as in an external Git client) and only use Domino to publish the Model API, the window will show a list of tools. Select from these tools to create a new workspace.
When you reproduce a workspace, as you did in the Reproduce the environment topic, Domino creates a branch in every repository involved in the project.
To publish a new model based on this reproduced branch, you must apply the commit to the master branch because Domino supports Git-based projects. The way that you do this depends on whether you are working with a Domino File System-based project or a Git-based project.
Commit projects based on the Domino File System (DFS):
-
Go to the Project.
-
Click Files. in the navigation bar.
-
From the Branch list, select the reproduced branch.
-
Click Revert Project to ensure that the commits made in this branch are added in the Master branch.
NoteRevert Project only works for the DFS files. For imported Git repositories or projects, you must revert the artifacts independently.
Commit Git-based projects:
-
In a Domino workspace or the Git tool of your choice, merge your latest code update into the master branch.
NotePerform the same merge operation for any artifacts (such as
.pklfiles) that you maintain in a separate repository. You can use Domino for the artifacts.-
In your Git-based project, in the navigation pane, click Artifacts.
-
Click Revert Project to restore the artifacts from your reproduced branch to the master branch.
-
Go to the Model API section of your project to publish a new Model API or a new version of an existing Model API. See Publish the model API.