API reference for the Agents Python SDK

The domino.agents package provides tracing, evaluation, and search capabilities for agentic systems. This is part of the python-domino library.

Note

The canonical import path is from domino.agents.tracing import …​
Some documentation may reference domino.aisystems.tracing. This is an older alias. Both paths resolve to the same module.

add_tracing

Decorator that starts an MLflow span for the decorated function. If an existing trace is in progress, appends a span to it; otherwise creates a new trace.

domino.agents.tracing.add_tracing(
    name: str,
    autolog_frameworks: list[str] | None = [],
    evaluator: Callable | None = None,
    trace_evaluator: Callable | None = None,
    eagerly_evaluate_streamed_results: bool = True,
    allow_tracing_evaluator: bool = False
)
ParameterTypeDescription

name

str

Name of the span (or new trace).

autolog_frameworks

list[str] | None

MLflow-supported frameworks to autolog (for instance, ["langchain"], ["openai"]).

evaluator

Callable | None

Function that receives the span and returns evaluation results as dict[str, int | float | str]. Runs on every invocation.

trace_evaluator

Callable | None

Function that receives the complete trace. Only fires when the trace was started and finished by this decorator. Useful for end-to-end quality checks.

eagerly_evaluate_streamed_results

bool

When True (default), aggregates all yielded values into a single span for evaluation. When False, each yielded value creates a separate span with a shared group_id.

allow_tracing_evaluator

bool

Default False. When True, inline evaluators will also be traced by MLflow autolog.

init_tracing

Initialize MLflow autologging and set the active experiment to enable tracing. Used to initialize logging in both development and production modes.

domino.agents.tracing.init_tracing(
    autolog_frameworks: list[str] | None = None
)

  • In production mode, the environment variables DOMINO_AGENT_IS_PROD and DOMINO_APP_ID must be set.

  • Call init_tracing() before your app starts serving requests.

search_traces

Search for traces in a development-mode evaluation run. Returns a paginated response of trace summaries.

domino.agents.tracing.search_traces(
    run_id: str,
    trace_name: str | None = None,
    start_time: datetime | None = None,
    end_time: datetime | None = None,
    page_token: str | None = None,
    max_results: int | None = None
) → SearchTracesResponse

search_agent_traces

Search for traces from a deployed production agent. Filter by agent version, trace name, and time range. If agent_version is not provided, searches across all versions.

domino.agents.tracing.search_agent_traces(
    agent_id: str,
    agent_version: str | None = None,
    trace_name: str | None = None,
    start_time: datetime | None = None,
    end_time: datetime | None = None,
    page_token: str | None = None,
    max_results: int | None = None
) → SearchTracesResponse

Data classes

EvaluationResult

class domino.agents.tracing.EvaluationResult(
    name: str,
    value: float | str
)
ParameterTypeDescription

name

str

Name of the evaluation.

value

float | str

Evaluation result value.

TraceSummary

class domino.agents.tracing.TraceSummary(
    name: str,
    id: str,
    spans: list[SpanSummary],
    evaluation_results: list[EvaluationResult]
)
ParameterTypeDescription

name

str

Name of the trace.

id

str

MLflow trace ID.

spans

list[SpanSummary]

Child spans of this trace.

evaluation_results

list[EvaluationResult]

Evaluation results for this trace.

SpanSummary

class domino.agents.tracing.SpanSummary(
    id: str,
    name: str,
    trace_id: str,
    inputs: Any,
    outputs: Any
)
ParameterTypeDescription

id

str

MLflow span ID.

name

str

Span name.

trace_id

str

Parent trace ID.

inputs

Any

Inputs to the function that created the span.

outputs

Any

Outputs of the function.

SearchTracesResponse

class domino.agents.tracing.SearchTracesResponse(
    data: list[TraceSummary],
    page_token: str | None
)
ParameterTypeDescription

data

list[TraceSummary]

List of trace summaries.

page_token

str | None

Token for the next page of results.

log_evaluation

Log an evaluation result against an existing trace. Used for ad-hoc or production evaluations where you evaluate traces after they’ve been collected such as in a scheduled Job.

domino.agents.logging.log_evaluation(
    trace_id: str,
    name: str,
    value: float | str
)
ParameterTypeDescription

trace_id

str

The MLflow trace ID to attach the evaluation to.

name

str

Name of the evaluation metric.

value

float | str

Evaluation result value (numeric score or string label).

DominoRun

The DominoRun wrapper from domino.agents.logging creates an MLflow experiment run that stores traces and logged parameters.

from domino.agents.logging import DominoRun

with DominoRun(
    agent_config_path="config.yaml",
    aggregated_metrics=[("toxicity", "mean"), ("bleu", "median")]
) as run:
    run_agent(...)
ParameterDescription

agent_config_path

Path to a YAML config file. Logged as parameters in the Experiment Manager.

aggregated_metrics

List of (metric_name, aggregation) tuples. Aggregation types: mean, median, stdev, min, max. Defaults to mean for all metrics.

Environment variables

VariableRequired forDescription

MLFLOW_TRACKING_URI

All modes

Set automatically in Domino executions. For local testing, use http://localhost:5000.

DOMINO_AGENT_IS_PROD

Production

Set to indicate production mode for init_tracing().

DOMINO_APP_ID

Production

The ID of the deployed agent app.