Agents · Benchmark profile
τ³-bench is the current evolution of Sierra's tool-agent-user framework, adding corrected task releases and newer knowledge and voice evaluation modes alongside airline, retail, and telecom.
Data verified 27 Jul 2026 · Methodology 1.8.0
Benchmark score on τ³-Bench Tool-Agent-User Evaluation
Mistral Medium 3.5 128B leads at 91.4%, followed by MiMo-V2.5-Pro (72.9%) and Nemotron 3 Ultra (70.9%).
10 ranked models · higher is better · labels show rank and score
| Model | Rank | Provider | Score |
|---|---|---|---|
| Mistral Medium 3.5 128B | #1 | Mistral AI | 91.4% |
| MiMo-V2.5-Pro | #2 | Xiaomi | 72.9% |
| Nemotron 3 Ultra | #3 | NVIDIA | 70.9% |
| Qwen3.6 Plus | #4 | Alibaba Cloud | 70.7% |
| GLM-5.1 | #5 | Z.AI | 70.6% |
| Claude Opus 4.5 | #6 | Anthropic | 70.2% |
| Qwen3.5 397B | #7 | Alibaba Cloud | 68.4% |
| Qwen3.6-35B-A3B | #8 | Alibaba Cloud | 67.2% |
| Kimi K2.5 | #9 | Moonshot AI | 65.7% |
| GLM-5 | #10 | Z.AI | 65.6% |
One best score per model · higher is better
| Rank | Model | Provider | License | Evidence use | Score |
|---|---|---|---|---|---|
| #1 | Mistral Medium 3.5 128B mistral-medium-3-5-128b | Mistral AI | open | Estimated reference | 91.4% |
| #2 | MiMo-V2.5-Pro mimo-v2-5-pro | Xiaomi | closed | Reference only | 72.9% |
| #3 | Nemotron 3 Ultra nemotron-3-ultra-550b | NVIDIA | open | Estimated reference | 70.9% |
| #4 | Qwen3.6 Plus qwen3-6-plus | Alibaba Cloud | closed | Reference only | 70.7% |
| #5 | GLM-5.1 glm-5-1 | Z.AI | open | Reference only | 70.6% |
| #6 | Claude Opus 4.5 claude-opus-4-5 | Anthropic | closed | Reference only | 70.2% |
| #7 | Qwen3.5 397B qwen3.5-397b-a17b | Alibaba Cloud | open | Estimated reference | 68.4% |
| #8 | Qwen3.6-35B-A3B qwen3-6-35b-a3b | Alibaba Cloud | open | Estimated reference | 67.2% |
| #9 | Kimi K2.5 kimi-k2.5 | Moonshot AI | open | Reference only | 65.7% |
| #10 | GLM-5 glm-5 | Z.AI | open | Reference only | 65.6% |
The top of this snapshot is led by Mistral Medium 3.5 128B at 91.4%; third place is 20.5 points behind. The top-10 spread is 25.8 points.
About τ³-Bench Tool-Agent-User Evaluation
τ³-bench is the current evolution of Sierra's tool-agent-user framework, adding corrected task releases and newer knowledge and voice evaluation modes alongside airline, retail, and telecom. Results stay tied to the exact model variant and evaluation system. Multiple systems for the same model use the best published score on this page; overall Lumina scoring uses the median of ranking-eligible rows.
Open benchmark source ↗FAQ
τ³-bench is the current evolution of Sierra's tool-agent-user framework, adding corrected task releases and newer knowledge and voice evaluation modes alongside airline, retail, and telecom.
Mistral Medium 3.5 128B by Mistral AI currently leads with 91.4%.
10 models in the LuminaBench cohort have a qualifying score on this benchmark.
Related
