Use the Admin Toolkit to run a system scan on your Domino deployment to do the following:

Check for unpatched known issues
Validate the health of core services
Get information to assist troubleshooting, run remediation actions, or provide context about issues affecting the deployment
Provide the toolkit report to Domino Support as part of an incident

To run the toolkit, you must be a Domino admin with access to the underlying Kubernetes cluster that hosts Domino. You can run the toolkit as many times as necessary because the results are read-only.

You can run the toolkit during any of these stages:

After a new Domino install
At the start of an incident
In the midst of an incident
After incident resolution to verify health
Before and after a Domino upgrade
Before and after a Domino migration

This tool relies on Kubernetes. Do not use it if the Kubernetes cluster is broken (such as network complications or the Kubernetes control plane is down).

You can find details about the specific tests that are included in the scan at https://docs.toolkit.re.domino.tech/.

Prerequisites

Bastion host, laptop, or other host with kubectl access that has network connectivity to the target cluster.

Use the following command to get the latest version of toolkit.sh:

curl -sSLO https://toolkit.re.domino.tech/toolkit.sh && chmod a+x ./toolkit.sh

Run the toolkit

Run the toolkit in a terminal.

Run the toolkit. This outputs a report in the console and uploads it to a location for Domino staff.
```
./toolkit.sh
```
This is the same as running toolkit.sh up && toolkit.sh test.
1. If the Domino installation blocks AWS access, run the following command to output the report only in the console:
  
  Note
  ./toolkit.sh up ./toolkit.sh pytest
2. If you do not have access to quay.io and mirrors.domino.tech, push the toolkit’s docker image to your internal registry.
  1. Run the following from the host where you have internet and internal docker registry access:
    
    ./toolkit.sh push internal-registry-host/domino-admin-toolkit
  2. From wherever you have kubectl access or from central, run the following:
    
    ./toolkit.sh up --image internal-registry-host/domino-admin-toolkit ./toolkit.sh pytest

Run the toolkit in a web interface

In the terminal, run the following command to get the URL:

$ ./toolkit.sh up
Web UI: https://<DOMINO_URL>/toolkit
Username: admin-toolkit, Password: PASSWORD (Valid till the pod is running)
Pod is Running ok!

Go to the URL provided.
Click Run.
When the report is ready, click the link to access the report.

View the reports

Use the Administrator’s toolkit web application to review the reports in HTML format. This application automatically terminates after one hour.

From wherever you have kubectl access, run the following:

./toolkit.sh up

The output is similar to the following:

Web UI: https://<DOMINO_URL>/toolkit
Username: admin-toolkit, Password: PASSWORD (Valid till the pod is running)
Pod is Running ok!

Go to the link specified in the web application output. If necessary, connect to the VPN.

`./toolkit.sh` usage

The following describes the toolkit parameters.

./toolkit.sh help

toolkit.sh version: 0d5903c, 2022-11-11

INFO: Checking for updates

Usage: toolkit.sh [command] [options]

Run with no parameters to create resources and run the tests

Commands:
  up     [options]  Create resources but don't run the tests
  test   [options]  Run tests with given options and upload html report to S3
  pytest [options]  Run pytest directly, see https://docs.pytest.org/en/7.1.x/how-to/usage.html
  clean             Delete resources
  exec [command]    Execute a command in the toolkit container (by default /bin/bash), useful for debugging
  logs              Show toolkit container logs
  help              Get this help message

`up` command options:
  ./toolkit.sh up --tag|-t tag              Run specific docker image tag of the admin toolkit
  ./toolkit.sh up --runtime-timeout timeout Set the maximum runtime of the toolkit in seconds. After this time the toolkit will automatically shutdown. Default is 3600
  ./toolkit.sh up --daemonset-port|-d port  Set the host port the daemonset listens on. Default is port 5000.
  ./toolkit.sh up --no-daemonset            Disable daemonset functionality
  ./toolkit.sh up --no-ingress              Disable ingress route to toolkit pod.

`test` command options:
  ./toolkit.sh test --help                  Show help including choice of parameters
  ./toolkit.sh test --exclude salt kubeadm  Exclude tests for specific features
  ./toolkit.sh test --log-cli-level DEBUG   Show extra debugging output
  ./toolkit.sh test --local-only            Copy HTML file to host and do not send the report output to Domino's S3

`pytest` command options:
  ./toolkit.sh pytest checks/<check>.py     Execute a single check

Changelog

November 2022

helm and yq dependencies are removed.
The web-based UI is now password protected.
The web-based UI can be disabled using --no-ingress parameter.

October 2022

Prerequisites now include helm and yq.
The web-based UI is now interactive.
The --load-balancer option is deprecated. Instead, use the new ingress route, /toolkit, on the same domain as the deployment.
A new --runtime-timeout option sets the maximum runtime of the toolkit in seconds.

Run a System Scan

Prerequisites

Run the toolkit

View the reports

./toolkit.sh usage

Changelog

`./toolkit.sh` usage