After deploying your agentic system to production, monitor its performance with real user interactions. Domino automatically collects production traces and provides dashboards to track performance, usage, and quality over time. Monitor agent performance Access monitoring views from your deployed agent’s dashboard. These views show how your agent performs with real user interactions.
-
Overview: Deployment status and configuration details
-
Performance: Evaluation metrics as visualizations alongside production traces. Review metrics over time to spot trends and identify patterns in successful versus problematic interactions
-
Usage: User invocations and interaction tracking
Run evaluations against production traces to continuously monitor your agent’s quality. Use the same evaluation methods you used during experimentation: manual UI annotations, inline evaluators, or adhoc evaluation scripts.
The most effective approach is to run evaluations as Domino Jobs. This lets you version and reproduce evaluations, and you can schedule Jobs to continuously monitor your agent’s quality as it handles user requests.
When you identify issues or opportunities for improvement, relaunch your production agent’s configuration into a workspace. Track and monitor experiments has details on relaunching runs.
This workflow lets you reproduce the exact production configuration, debug issues identified in production traces, and maintain clear lineage between production agents and their source experiments.
-
Develop agentic systems: Iterate on your agent configuration based on production insights
-
Experiment tracking and traces with agents: Test improvements before redeploying