Domino Governance policy components

Policies can be managed in the Governance Console.

Once a policy is published, it becomes immutable, ensuring the lineage between a Governed Bundle and the approved policy can be maintained.

Note
 To complete these tasks, you’ll need to be assigned the GovernanceAdmin role. CloudAdmins and SysAdmins already have permissions associated with Domino Governance.

Define Policies with YAML

This document explains the YAML configuration structure that defines various input fields, policy scripted checks, metadata, and guidance elements.

Structure overview

The YAML configuration consists of a list of items, each representing a different element in the user interface or system configuration.

Each item has the following common properties:

  • artifactType: Defines the type of artifact (e.g., input, policyScriptedCheck, metadata, guidance).

The details section contains specific properties for each artifact type.

Stages

A stage contains a group of evidence and approvals. Direct evidence is grouped as an evidence set, and approvals are grouped as approvals. Each approval includes a name, a list of approvers, and optionally, evidence.

yaml
stages:
  - name: stage1
  - name: stage2

  • name: The name of the stage.

Evidence and evidence sets

Evidence includes inputs, approvals, and checks that should be gathered as part of the governance process within a given stage. Evidence sets represent logical groupings of evidence and can also be reused in other policies if needed.

yaml
evidenceSet:
  - id: Local.sample
    name: sample local evidence
    description: Describe the sample local evidence
    definition: Define the evidence

  • Local: Local evidence should have the full definition included when appearing for the first time in the policy. It can later be referenced by id when using it in the YAML file.

  • name: Name of the evidence.

  • description: This is where the evidence is described.

  • definition: This is where artifacts are defined. Guidance artifact types must have a text field under details. Input artifact types must have type and label fields. Each artifact contains one UI element. Supported types include radio, textinput, textarea, select, multiselect, checkbox, date, and numeric.

Metrics checks

Model metrics in these cases refer to a set of metrics encapsulated by the policy YAML and there can be 1+n model metrics. Users can then add additional metrics to a set of model metrics and have multiple sets of model metrics. Each model metrics set will have the same required fields defined by the policy as a baseline. Model metrics provide automated checks for pre-approval to lighten the burden of governance admins.

yaml
metrics:
- id: Local.model-quality
  name: Model Quality
  description: Describe the model quality
  definition:
    - artifactType: metadata
      details:
      type: modelmetric
      metrics:
        - name: Acc
          label: Accuracy
          aliases:
            - acc
            - Correct Classification Rate
            - Percentage Correct
          threshold:
            operator: '>='
            value: 0.8

  • Local: Local evidence should have the full definition included when appearing for the first time in the policy. It can later be referenced by id when using it in the YAML file.

  • name: Name of the model metrics.

  • description: This is where the model quality is described.

  • definition: This is where model metrics are defined.

  • label: How the metric name appears in the evidence notebook.

  • aliases: Alternatives to name metrics, to be used in the search to automatically add the metric to the notebook.

  • threshold: Operations to define a threshold for the metric include operators such as equal "=", greater than ">", and less than "<", among other relational operators.

  • value: Represents the actual threshold.

Scripted checks

These checks use centralized scripts, as defined in the policy YAML, to evaluate information on a Governed Bundle. Scripted checks can be used to measure bias consistently in a dataset.

yaml
    - artifactType: policyScriptedCheck
      details:
        name: Ethic and Fairness Evaluation
        label: Ethic and Fairness Evaluation
        command: evaluate_model.py create --model-hub ${model_hub} --model-name ${model_name}
        parameters:
          - name: model_hub
            type: text
            default: openai
          - name: model_name
            type: text
            default: gpt-4
            outputTypes:
              - txt
              - png
            environmentId: 674f04e2191e8f19a5d12552   # 6.0 default environment on se-demo
              hardwareTierId: small-k8s
              volumeSizeGiB: 4
    - artifactType: metadata
      details:
        label: Upload model validation report.
        type: file

  • Local: Local evidence should have the full definition included when appearing for the first time in the policy. It can later be referenced by id when using it in the YAML file.

  • name: Name of the scripted check.

  • description: This is where the scripted check is described.

  • definition: This is where scripted checks are defined.

  • label: How the script name appears in the evidence notebook.

  • aliases: Alternatives to name, to be used in the search to automatically add the metric to the notebook.

  • threshold: Operations to define a threshold for the metric include operators such as equal "=", greater than ">", and less than "<", among other relational operators.

  • value: Represents the actual threshold.

Input Artifacts

Input artifacts represent various form elements for user input.

Radio buttons

yaml
- artifactType: input
  details:
    type: radio
    label: "How would you rate the model risk?"
    options:
      - label: "High"
        value: "High"
      - label: "Medium"
        value: "Medium"
      - label: "Low"
        value: "Low"
    tooltip: "Guidance text"

  • type: radio: Defines a set of radio buttons.

  • label: The question or prompt for the radio group.

  • options: A list of choices, each with a label (displayed text) and value (submitted data).

  • tooltip: Additional information shown on hover.

Text input

yaml
- artifactType: input
  details:
    type: textinput
    label: "What are the expected business benefits?"
    placeholder: "Explain the benefit"
    helpText: "The text under the input box to help the user"

  • type: textinput: Defines a single-line text input field.

  • label: The prompt for the input.

  • placeholder: Example text is shown when the field is empty.

  • helpText: Additional guidance is displayed below the input field.

Text area

yaml
- artifactType: input
  details:
    type: textarea
    label: "What are the expected business benefits?"
    height: 10
    placeholder: "Explain the benefit"
    helpText: "The text under the input box to help the user"

  • type: textarea: Defines a multi-line text input field.

  • height: Specifies the height of the text area in lines.

Select dropdown

yaml
- artifactType: input
  details:
    type: select
    label: "Please select the base model template."
    options:
      - label: "base model1"
        value: "baseModel1"
      - label: "base model2"
        value: "baseModel2"

  • type: select: Defines a dropdown selection field.

  • options: A list of choices, similar to radio buttons.

Multi-select

yaml
- artifactType: input
  details:
    type: multiSelect
    label: "Please select the data sets used in the model."
    options:
      - label: "data set1"
        value: "dataset1"
      - label: "data set2"
        value: "dataset2"
      - label: "data set3"
        value: "dataset3"

  • type: multiSelect: Defines a multi-select dropdown field and allows selection of multiple options.

Checkbox group

yaml
- artifactType: input
  details:
    type: checkbox
    label: "Please select the departments that will use the model?"
    options:
      - label: "Sales"
        value: "DEPT001"
      - label: "Customer Success"
        value: "DEPT002"

  • type: checkbox: Defines a group of checkboxes and allows selection of multiple options.

Date input

yaml
- artifactType: input
  details:
    type: date
    label: "What is the scheduled release date?"
    startDate: 20240612
    format: ISO8601

  • type: date: Defines a date input field.

  • startDate: Sets the initial date (format: YYYYMMDD).

  • format: Specifies the date format (e.g., ISO8601).

Numeric input

yaml
- artifactType: input
  details:
    type: numeric
    label: "What is the allowed F score for the model to be deployed?"
    min: 0
    max: 1

  • type: numeric: Defines a numeric input field.

  • min and max: Set the allowed range for the input.

Guidance artifacts

Guidance artifacts provide guidance or information to the user.

yaml
- artifactType: guidance
  details:
    type: textblock
    text: >-
      [Map 1.4](https://ournistpolicyreferenceurl.com) The business value or
      context of business use has been clearly defined or - in the case of
      assessing existing AI systems - re-evaluated

  • type: textblock: Displays a block of text.

  • text: The content to display - supports Markdown formatting.

yaml
- artifactType: guidance
  details:
    type: banner
    text: >-
      [Map 1.4](https://ournistpolicyreferenceurl.com) The business value or
      context of business use has been clearly defined or - in the case of
      assessing existing AI systems - re-evaluated

  • type: banner: Displays a prominent banner with text.

  • Useful for important notices or key information

Approvals

Approvals are defined under stages. Each approval is defined with a name, a group of specified approvers, and evidence. Approvers must be Domino users or organizations and are specified by the user’s or organization’s name.

yaml
- name: 'Stage 4: validation sign off'
  approvers:
    - model-gov-org
  evidence:
    id: Local.validation-approval-body
    name: Sign-off
    description: The checklist for approvals
    definition:
      - artifactType: input
        details:
          label: "Have you read the model validation reports?"
          type: radio
          options:
            - Yes
            - No

Classification

Classification is a top-level variable often used to classify a Governed Bundle in a risk tier (low, medium, or high).

yaml
classification:
  rule:
  artifacts:
    - model-risk

stages:
  - name: classificationExample
      - id: Local.model-risk
        name: Model Risk
        description: Describe the risk of the model
        definition:
          - artifactType: input
            aliasForClassification: model-risk
            details:
              label: "How would you rate the model risk?"
              type: radio
              options:
                - High
                - Low
              tooltip: guidance on how to rate the model risk within the organization

  • classification: Policy-level classification.

  • rule: Classification rule definition; leave it empty if using the result of the artifact defined below.

  • artifacts: The artifact alias that is specified in the evidence definition.

  • aliasForClassification: Reference to classification artifacts.

Classification rule

Classification rules allow the construction of complex operations based on the outcome of multiple pieces of evidence.

func() string {
    var sum float64
    for _, value := range inputs {
        sum += value
    }
    if sum >= 1 {
        return "High"
    }
    return "Low"
}()

Visibility Rules

Visibility rules apply at an evidence-set level. Based on the visibility rule, we can customize whether to display a set of questions. For example, some evidence might be required only for Governed Bundles classified as high-risk.

yaml
evidenceSet:
- id: Global.type-of-development
    visibilityRule: inputs=="High"

  • visibilityRule: Optional, the visibility rule shows or hides evidence based on the classification value. In this example, the evidence set will only be displayed if the classification is High.