LuminaBench
Mathematics benchmark / MathArena / ETH SRI

ArXivLean — September 2026

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Accuracy

Version MathArena-arxivlean--september-20261106Verified 6 November 20267 published resultsOfficial source

ArXivLean — September 2026

LuminaBench
Show
7 of 7 models · best variant per model
03006009001,2001,093Claude Opus 5.5High1,059Gemini 4 Argon1,051GPT-6 AstraMax1,042GPT-6.1 SolMax958Kimi K3.1 Pro907DeepSeek V4.1 FlashMax890Qwen3.8 2.4T A95B

Lumina Score vs Cost per Task

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted.

LuminaBench
7 of 7 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.$1$3$10$308509009501,0001,0501,100Cost per task · USD (log scale)Lumina ScoreClaude Opus 5.5GPT-6.1 SolQwen3.8 2.4T A95BDeepSeek V4.1 FlashGemini 4 ArgonGPT-6 AstraKimi K3.1 Pro

Lumina Score vs Tokens

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted.

LuminaBench
7 of 7 families
100K300K1M8509009501,0001,0501,100Output tokens per task (log scale)Lumina ScoreClaude Opus 5.5GPT-6 AstraQwen3.8 2.4T A95BDeepSeek V4.1 FlashGemini 4 ArgonGPT-6.1 SolKimi K3.1 Pro

Lumina Score vs Time per Task

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted.

LuminaBench
7 of 7 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.01002003004008509009501,0001,0501,100Time per task · minLumina ScoreGPT-6.1 SolGPT-6 AstraClaude Opus 5.5Qwen3.8 2.4T A95BDeepSeek V4.1 FlashGemini 4 ArgonKimi K3.1 Pro

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Method & coverage

September 2026 Lean formal proof construction; preserve model configuration, native effort, warning flags and monthly task cohort. No visual-math competition imported.

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted.

How the Lumina conversion works

1,000 + 75 × ((original score − 26.41571428571428) / 16.663397747452226). The result is rounded once. Scores may tie.

Frozen initial observed text-only cohort; one mean per exact publisher release/API, balancing repeated settings/routes within that release. Original score and protocol remain separate. No inferred variants or future refitting.

Calibration
matharena-lean-september-z75-v1 · experimental
Reference mean
26.41571428571428
Population SD
16.663397747452226
Coverage
Standalone benchmark.
Identity, exclusions & attribution

Original measurements and exact source settings retained; unknown API, effort and harness versions remain unknown.

September 2026 Lean formal proof construction; preserve model configuration, native effort, warning flags and monthly task cohort. No visual-math competition imported.

Standalone text benchmark; no new combined index. Source identity/effort/protocol qualifications remain in Data and CSV.

All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.

Official source

Publisher terms. Scores and resource figures are attributed to the original publisher.