Develop agentic systems

Instrument your agentic system to capture traces during execution. Tracing records every call your system makes, including downstream calls from agents and tools, so you can evaluate performance and debug issues. To develop an agentic system in Domino:

To develop an agentic system in Domino:

  1. Instrument your code to capture traces

  2. Run your system and collect trace data

  3. Review and evaluate the traces

This page covers instrumentation - preparing your code to capture traces during execution.

Prerequisites

  • Familiarity with MLflow runs and the Domino Experiment Manager.

  • Domino Python SDK and mlflow==3.2.0. installed in your environment.

To install these packages, add the following to your Dockerfile instructions:

RUN pip install --no-cache-dir "git+https://github.com/dominodatalab/python-domino.git@master#egg=dominodatalab[data,aisystems]"
RUN pip install mlflow==3.2.0
Note
 Domino Standard Environments (DSEs) currently include an older MLflow version and do not include the Domino SDK. This requirement is temporary and will be removed once SDK support is built into DSE.

Step 1: Add the tracing decorator

Add the @add_tracing decorator to your functions that invoke your system or agent. This captures every call, including downstream calls, and logs them in your experiment run.

Domino supports all MLflow auto tracing integrations, such as LangChain, OpenAI, Pydantic AI, and others. Parameters, metrics, and artifacts from your runs are automatically captured alongside trace data.

If you don’t use an autolog framework, the traced function will still add a span to an existing trace or create a new one if none is in progress. This lets you capture trace data even when you aren’t working with a specific agent framework.

Example (using langchain):

from domino.agents.tracing import add_tracing

@add_tracing(name="prioritize_ticket", autolog_frameworks=["langchain"])
def run_agent(...):
...

Example (framework-agnostic):

from domino.agents.logging import DominoRun

with DominoRun() as run:
    run_agent(...)

Step 2: Run with DominoRun() wrapper

After decorating your function, run it inside an MLflow run using the DominoRun() wrapper. This creates an experiment run that stores your traces and any logged parameters or metrics.

Add YAML configuration (optional)

Pass a YAML configuration file into DominoRun() to log your system configuration as parameters in the Experiment Manager.

with DominoRun(agent_config_path=CONFIG_PATH) as run:

Add aggregated metrics (optional)

Provide a list of metric-aggregation tuples when creating a DominoRun to make it easier to compare runs. For example:

metrics = [("toxicity_score", "mean"), ("bleu_score", "median")]
run = DominoRun(..., aggregated_metrics=metrics)

Domino computes these aggregated metrics and attaches them to the corresponding run. You can specify the following aggregation types: mean, median, stdev, min, or max.

If you don’t provide aggregated_metrics, Domino automatically logs the mean of every metric logged to traces in the run.

Next steps