Monitor Data Source logs

Domino automatically creates an audit trail for Domino Data Source activity that records the WHO, WHERE, WHEN & WHAT for user activity. Audit trails can provide a means to help accomplish several security-related objectives, including the following:

  • Individual accountability

  • Reconstruction of events or security breaches

  • Intrusion Detection

  • Problem Analysis

  • Ensuring compliance with industry regulations

Tip
 We recommend you use at least dominodatalab-data==5.7.2 PyPI version for the most thorough metadata event tracking. For R users, we recommend the latest 0.2.4 DominoDataR release.

Domino logs the following Data Source events:

  • Create Data Sources

  • Delete Data Sources

  • Edit Data Source permissions

  • Edit Data Source ownership

  • Access Data Sources from Domino executions

  • Add Data Sources to projects

  • Remove Data Sources from projects

Note
 Data Source access events are logged for all executions (Workspaces, Jobs, Scheduled Jobs, Apps, and Launchers), except Domino endpoints.

Access the audit logs via the web UI or Public REST API endpoint, /api/datasource/v1/audit.

Data Source API

Use the Data Source audit Public REST API endpoint, /api/datasource/v1/audit, which takes several optional filter parameters, to access Data Source logs programmatically.

All requests must contain the header X-Domino-Api-Key with a value corresponding to a SysAdmin API key.

Unfiltered Data Source audit API sample

The following is a request with no filter parameters, it returns all events within the last 24 hours.

// request
https://<domino-url>/api/datasource/v1/audit

// response: with no filter parameters, all events in the last 24 hours are returned
[
    {
        "dataSourceId": "64e67c6005e8b2388f992545",
        "dataSourceName": "snowflake-ds",
        "dataSourceType": "SnowflakeConfig",
        "eventKind": "DeleteDataSource",
        "metadata": {
            "performedByUsername": "sample-admin"
        },
        "performedBy": "64e6363605e8b2388f9924c8",
        "timestamp": "2023-08-23T21:40:21.669Z"
    },
    {
        "dataSourceId": "64e67c6005e8b2388f992546",
        "dataSourceName": "s3-ds",
        "dataSourceType": "S3Config",
        "eventKind": "AccessDataSource",
        "metadata": {
            "runType": "Workspace",
            "projectId": "64e6371205e8b2388f9924d4",
            "dataPlaneId": "000000000000000000000000",
            "performedByUsername": "sample-user",
            "runId": "64e6632005e8b2388f9924eb"
        },
        "performedBy": "64e6371105e8b2388f9924d2",
        "timestamp": "2023-08-21T21:30:05.439Z"
    }
]

Filtered Data Source audit API sample

The following is a request with several filter parameters, including startTime, endTime, multiple dataSourceNames, and eventKinds, for a more granular response.

// request
https://<domino-url>/api/datasource/v1/audit/?startTime=2022-08-17T23:09:24.921Z&endTime=2023-08-25T23:09:24.921Z&dataSourceNames=snowflake-ds&dataSourceNames=s3-ds&eventKinds=AccessDataSource

// response
[
    {
        "dataSourceId": "64e67c6005e8b2388f992546",
        "dataSourceName": "s3-ds",
        "dataSourceType": "S3Config",
        "eventKind": "AccessDataSource",
        "metadata": {
            "runType": "Workspace",
            "projectId": "64e6371205e8b2388f9924d4",
            "dataPlaneId": "000000000000000000000000",
            "performedByUsername": "sample-user",
            "runId": "64e6632005e8b2388f9924eb"
        },
        "performedBy": "64e6371105e8b2388f9924d2",
        "timestamp": "2023-08-21T21:30:05.439Z"
    },
    {
        "dataSourceId": "64e67c6005e8b2388f992545",
        "dataSourceName": "snowflake-ds",
        "dataSourceType": "SnowflakeConfig",
        "eventKind": "AccessDataSource",
        "metadata": {
            "runType": "Workspace",
            "projectId": "64e6371205e8b2388f9924d4",
            "dataPlaneId": "000000000000000000000000",
            "performedByUsername": "sample-user",
            "runId": "64e6632005e8b2388f9924eb"
        },
        "performedBy": "64e6371105e8b2388f9924d2",
        "timestamp": "2023-02-05T21:40:01.449Z"
    },
]

Next steps

See Project audit logs for information on project-level auditing including user permissions and Dataset-related actions.