A task with its own rules.
Measures multi-step financial research and evidence synthesis by an agent system.
- Organisation
- Version
- v2
Loading benchmark header
Loading benchmark registry
Research / Benchmark profile
Measures multi-step financial research and evidence synthesis by an agent system.
Observed results
5 ranked models · higher is better · labels show rank and score
| Model | Rank | Model ID | Provider | Score | Unit |
|---|---|---|---|---|---|
| Gemini 3.5 Flash | #1 | gemini-3-5-flash | 57.9 | percent | |
| Muse Spark 1.1 | #2 | muse-spark-1-1 | Meta | 57.2 | percent |
| Claude Opus 4.8 | #3 | claude-opus-4-8 | Anthropic | 53.9 | percent |
| GPT-5.5 | #4 | gpt-5-5 | OpenAI | 51.8 | percent |
| Gemini 3.1 Pro Preview | #5 | gemini-3-1-pro-preview | 43 | percent |
One best compatible score per canonical product · higher is better
| Rank | Model | Weights | Evidence | Score |
|---|---|---|---|---|
| 01 | Gemini 3.5 FlashGoogle | Closed weights | Public reference | 57.9% |
| 02 | Muse Spark 1.1Meta | Closed weights | Public reference | 57.2% |
| 03 | Claude Opus 4.8Anthropic | Closed weights | Source carrier | 53.9% |
| 04 | GPT-5.5OpenAI | Closed weights | Public reference | 51.8% |
| 05 | Gemini 3.1 Pro PreviewGoogle | Closed weights | Public reference | 43% |
Each distinct published measurement is retained. Same-snapshot canonical and configuration projections appear once.
| Published model / configuration | Score | Evidence & protocol | Source & dates |
|---|---|---|---|
Exact BenchLM registry variant Gemini 3.5 Flash; bulk export does not retain a complete upstream harness configuration.Gemini 3.5 FlashExact identitySource label without a registered configuration ID Canonical product: gemini-3-5-flash | 57.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Muse Spark 1.1; bulk export does not retain a complete upstream harness configuration.Muse Spark 1.1Exact identitySource label without a registered configuration ID Canonical product: muse-spark-1-1 | 57.2% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Claude Opus 4.8; bulk export does not retain a complete upstream harness configuration.Claude Opus 4.8Exact identitySource label without a registered configuration ID Canonical product: claude-opus-4-8 | 53.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-08-01Observed Checked |
Exact BenchLM registry variant Gemini 3.5 Flash; bulk export does not retain a complete upstream harness configuration.Gemini 3.5 FlashExact identitySource label without a registered configuration ID Canonical product: gemini-3-5-flash | 57.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-07-27Observed Checked |
Exact BenchLM registry variant Muse Spark 1.1; bulk export does not retain a complete upstream harness configuration.Muse Spark 1.1Exact identitySource label without a registered configuration ID Canonical product: muse-spark-1-1 | 57.2% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-07-27Observed Checked |
Exact BenchLM registry variant Claude Opus 4.8; bulk export does not retain a complete upstream harness configuration.Claude Opus 4.8Exact identitySource label without a registered configuration ID Canonical product: claude-opus-4-8 | 53.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 2026-07-27Observed Checked |
Exact BenchLM registry variant Gemini 3.5 Flash; bulk export does not retain a complete upstream harness configuration.Gemini 3.5 FlashExact identitySource label without a registered configuration ID Canonical product: gemini-3-5-flash | 57.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 21 July 2026Observed Checked |
Exact BenchLM registry variant Muse Spark 1.1; bulk export does not retain a complete upstream harness configuration.Muse Spark 1.1Exact identitySource label without a registered configuration ID Canonical product: muse-spark-1-1 | 57.2% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 21 July 2026Observed Checked |
Exact BenchLM registry variant Claude Opus 4.8; bulk export does not retain a complete upstream harness configuration.Claude Opus 4.8Exact identitySource label without a registered configuration ID Canonical product: claude-opus-4-8 | 53.9% | Source carriersource-checkedVersion & system2026 Source-native system | BenchLM public datasets — 21 July 2026Observed Checked |
Muse Spark 1.1; provider evaluation configuration as published in the Meta Muse Spark 1.1 evaluation report (Finance Agent v2).Muse Spark 1.1Exact identitymuse-spark-1-1-xhigh Canonical product: muse-spark-1-1 | 57.2% | Public referenceprovider-reportedVersion & systemv2 Provider evaluation harness | Meta AI Muse Spark 1.1 evaluation reportObserved Checked |
From result to context
Measures multi-step financial research and evidence synthesis by an agent system.
The summary shows one best compatible source result per canonical product. Versions, effort settings and execution systems remain attached to the underlying records.
Scores from different versions or harnesses may not be interchangeable. The published source rows preserve those distinctions and their original units. A source result is not automatically an input to a current index.
This catalogue entry preserves available source evidence. Current indices admit only their specifically reviewed tracks and configurations.
Contamination risk: Unknown. Lifecycle: Active.