Use Model Monitoring APIs

In a production setting, you can automate some of the repetitive tasks through APIs. Following is the list of available APIs.

List Models

This API enables users to list all the models registered with the Model Monitor within the user’s organization.

Request Format:

GET  `/api/v0/models/list_models`

Sample Request:

curl --location --request GET 'https://develop.dmm.domino.tech/api/v0/models/list_models'
--header 'Authorization: <API-TOKEN>'
--header 'Content-Type: application/json'
--data-raw ''

Sample Response:

[
    {
        "id": "5eb505fc984c0610c5673653",
        "name": "PropensityPrediction",
        "modelType": "classification",
        "modelVersion": "1",
        "description": "Propensity to purchase prediction",
        "dateRegistered": "2020-05-08T07:10:52.471000",
        "modelAuthor": "samit.thange@dominodatalab.com"
    },
]

Register Model

This API registers a new model for monitoring. It takes a valid Model Config JSON file as input.

Request Format:

POST  `/api/v0/models/register_model`

Sample request:

curl --location --request POST 'https://develop.dmm.domino.tech/api/v0/models/register_model'
--header 'Authorization: <API-TOKEN>'
--header 'Content-Type: application/json'
--data-raw '{
    "variables": [
        {
            "name": "age",
            "valueType": "numerical",
            "variableType": "feature"
        },
        {
            "name": "y",
            "valueType": "categorical",
            "variableType": "prediction"
        },
        {
            "name": "date",
            "valueType": "datetime",
            "variableType": "timestamp"
        },
        {
            "name": "RowId",
            "valueType": "string",
            "variableType": "row_identifier"
        }
    ],
    "datasetDetails": {
        "name": "BMAF-TrainingData-Webinar.csv",
        "datasetType": "file",
        "datasetConfig": {
            "path": "BMAF-TrainingData-Webinar.csv",
            "fileFormat": "csv"
        },
        "datasourceName": "dmm-shared-bucket",
        "datasourceType": "s3"
    },
    "modelMetadata": {
        "name": "test_psg",
        "modelType": "classification",
        "version": "2",
        "description": "",
        "author": "testadmin"
    }
}'

Sample Response:

{
    "id": "5ec541a54ec87ed79851d121",
    "name": "API-Test",
    "modelType": "classification",
    "modelVersion": "1",
    "description": "API tests",
    "dateRegistered": "2020-05-20T14:41:41.448000",
    "modelAuthor": "samit.demo@dominodatalab.com"
}

Add Prediction Data

This API adds prediction data to a model.

Request Format:

PUT  `/api/v0/models/<modelID>/add_predictions`

Sample Request:

curl --location --request PUT 'https://develop.dmm.domino.tech/api/v0/models/5ec541a54ec87ed79851d121/add_predictions'
--header 'Authorization: <API-TOKEN>'
--header 'Content-Type: application/json'
--data-raw '{
    "variables": [
        {
            "name": "education",
            "variableType": "sample_weight",
            "valueType": "numerical"
        }
    ],
    "datasetDetails": {
        "name": "BMAF-Predictions-Webinar.csv",
        "datasetType": "file",
        "datasetConfig": {
            "path": "BMAF-Predictions-Webinar.csv",
            "fileFormat": "csv"
        },
    "datasourceName": "dmm-shared-bucket",
    "datasourceType": "s3"
    }
}'

Sample Response:

{
    "status": "success"
}

Add Ground Truth Data

This API adds Ground Truth data to a model.

Request Format:

PUT  `/api/v0/models/<modelID>/add_ground_truths`

Sample Request:

curl --location --request PUT 'https://develop.dmm.domino.tech/api/v0/models/5ec541a54ec87ed79851d121/add_ground_truths'
--header 'Authorization: <API-TOKEN>'
--header 'Content-Type: application/json'
--data-raw '{
    "variables": [
        {
            "name": "y_gt",
            "variableType": "ground_truth",
            "valueType": "categorical",
            "forPredictionOutput": "y"
        }
    ],
    "datasetDetails": {
        "name": "BMAF-GTLabels-Webinar.csv",
        "datasetType": "file",
        "datasetConfig": {
            "path": "BMAF-GTLabels-Webinar.csv",
            "fileFormat": "csv"
        },
        "datasourceName": "dmm-shared-bucket",
        "datasourceType": "s3"
    }
}'

Sample Response:

{
    "status": "success"
}

Unregister Model

This API unregisters the model from monitoring.

Request Format:

DELETE  `/api/v0/models/<modelID>`

Sample request:

curl --location --request DELETE 'https://develop.dmm.domino.tech/api/v0/models/5ec541a54ec87ed79851d121'
--header 'Authorization: <API-TOKEN>'
--data-raw ''

Sample Response:

{
    "status": "success"
}