Why this matters Most public LLM datasets focus on prompts, responses or evaluation labels; observability data that records how agents and models behave in production-style runs is far rarer. This dataset fills that gap by providing structured OpenTelemetry traces that capture the timing, token usage, model choices, and tool invocations for real multi-step agent executions—data you can use to quantify latency, resource patterns, failure modes, and tool-usage strategies.
What Sets It Apart
- Execution-level observability: each trace is a full OpenTelemetry span tree (start/end timestamps, span names, parent relationships) rather than isolated prompt/response pairs. That makes it possible to measure fine-grained latencies (per-call and per-tool) and to reconstruct multi-call workflows.
- Rich metadata per call: traces include input/output token counts, model identifiers, finish reasons, response IDs, tool definitions and tool call arguments/results — enabling cross-model cost and behavior comparisons without re-running workloads.
- Multi-benchmark, multi-framework coverage: 1,781 traces across six benchmarks (retail, airline, telecom, software-engineering, AppWorld, BrowseCompPlus) and five agent frameworks, which helps surface pattern generality vs. benchmark-specific behaviors.
Who it's for & trade-offs
Great fit if you want to: compare inference latency and token usage across models/providers; analyze how agents partition work between LLM calls and external tools; detect recurring failure states and their preconditions. Look elsewhere if you need raw user telemetry (PII) or massive-scale production traces: this collection is mid-sized (1.7k traces) and curated for reproducible analysis rather than exhaustive fleet monitoring. Also note that some model names in traces reference closed-source provider labels; licensing and reproduction of exact model behavior may be limited.
Where it fits
Use this dataset for offline research (profiling, benchmarking, anomaly detection), benchmarking tools that orchestrate LLMs, or as a labeled source for building models that predict costly or failing inference patterns. It’s complementary to token-level corpora: token data here is contextualized by execution structure and tool usage, which is essential for engineering optimizations and MLOps workflows.
