LuminaBench
Coding benchmark / Epoch AI

MirrorCode

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Solve rate

Version ML-private-2-languages-30-tasks-v1Verified 6 November 202615 published resultsOfficial source

MirrorCode

LuminaBench
Show
15 of 15 models · best variant per model
03006009001,2001,137Claude Opus 5.5High1,119Claude Fable 5.1High1,088Claude Fable 5High1,072Gemini 4 ArgonHigh1,032GPT-6 AstraHigh1,014Gemini 3.7 FlashHigh1,014Grok 4.6xHigh985Grok 4.5High981claude-opus-4-7High967GPT-6 LunaMax945GPT-5.6 SolHigh930GPT-5.4High912GPT-5.5High909gemini-3.1-pro-previewHigh894GPT-5High

No comparable per-task resource measurements were published. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Method & coverage

MirrorCode leaderboard ML +Private 2L: 15 programs in two implementation languages, 30 tasks; three attempts, 10 billion tokens and seven days per attempt. Distinct from the paper protocol.

No comparable per-task resource measurements were published.

How the Lumina conversion works

1,000 + 75 × ((original score − 36.851925925925926) / 22.941096161276445). The result is rounded once. Scores may tie.

Frozen initial observed text-only cohort; one mean per exact publisher release/API, balancing repeated settings/routes within that release. Original score and protocol remain separate. No inferred variants or future refitting.

Calibration
mirrorcode-z75-v1 · experimental
Reference mean
36.851925925925926
Population SD
22.941096161276445
Coverage
Standalone benchmark.
Identity, exclusions & attribution

MirrorCode leaderboard ML +Private 2L: 15 programs in two implementation languages, 30 tasks; three attempts, 10 billion tokens and seven days per attempt. Distinct from the paper protocol.

Each run ID is retained. Repeated observations are not additional model releases or native reasoning settings. Standard errors are retained only where their mean-score association matches the selected primary metric.

Text mathematical statements/proof/code tools; no image, video, speech or screenshot input protocol.

Standalone text benchmark; no new combined index. Source identity/effort/protocol qualifications remain in Data and CSV.

All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.

Official source · Publication terms / evidence

Publisher terms. Scores and resource figures are attributed to the original publisher.