LuminaBench
Mathematics benchmark / MathArena / ETH SRI

ArXivMath — September 2026

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Accuracy

Version MathArena-arxiv--september-20261106Verified 6 November 202612 published resultsOfficial source

ArXivMath — September 2026

LuminaBench
Show
12 of 12 models · best variant per model
03006009001,2001,132GPT-6.1 SolMax1,117Gemini 4 Argon1,094Claude Opus 5.5High1,031Muse Wonder1,023Kimi K3.1 Pro967Gemini 3.8 Flash964DeepSeek V4.1 Flash 1017Max949Grok 4.7xHigh949Qwen3.8 2.4T A95B949DeepSeek-V4.1-Flash-1017 (Max, DSH)930DeepSeek V4.1 FlashMax896Muse Spark Open

Lumina Score vs Cost per Task

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted.

LuminaBench
12 of 12 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.$0.3$1$3$109001,0001,1001,200Cost per task · USD (log scale)Lumina ScoreDeepSeek-V4.1-Flash-1017 (Max, DSH)GPT-6.1 SolGemini 4 ArgonClaude Opus 5.5Muse WonderKimi K3.1 Pro

Lumina Score vs Tokens

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted.

LuminaBench
12 of 12 families
0200K400K600K9001,0001,1001,200Output tokens per taskLumina ScoreClaude Opus 5.5GPT-6.1 SolGemini 4 ArgonGemini 3.8 FlashKimi K3.1 ProMuse Wonder

Lumina Score vs Time per Task

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted.

LuminaBench
12 of 12 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.0501001509001,0001,1001,200Time per task · minLumina ScoreGemini 3.8 FlashGPT-6.1 SolGemini 4 ArgonClaude Opus 5.5Muse WonderKimi K3.1 Pro

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Method & coverage

September 2026 text mathematical problems from recent arXiv research; preserve model configuration, native effort, warning flags and monthly task cohort. No visual-math competition imported.

Original mean per-problem costs, token counts and time with monthly cohort; no total/budget values substituted.

How the Lumina conversion works

1,000 + 75 × ((original score − 58.25583333333333) / 18.91622516004595). The result is rounded once. Scores may tie.

Frozen initial observed text-only cohort; one mean per exact publisher release/API, balancing repeated settings/routes within that release. Original score and protocol remain separate. No inferred variants or future refitting.

Calibration
matharena-september-z75-v1 · experimental
Reference mean
58.25583333333333
Population SD
18.91622516004595
Coverage
Standalone benchmark.
Identity, exclusions & attribution

Original measurements and exact source settings retained; unknown API, effort and harness versions remain unknown.

September 2026 text mathematical problems from recent arXiv research; preserve model configuration, native effort, warning flags and monthly task cohort. No visual-math competition imported.

Standalone text benchmark; no new combined index. Source identity/effort/protocol qualifications remain in Data and CSV.

All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.

Official source

Publisher terms. Scores and resource figures are attributed to the original publisher.