LuminaBench
Reasoning benchmark / Epoch AI

SimpleQA Verified

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Proportion correct on 1,000 verified factual questions. Epoch AI evaluations.

Version Verified · 27 Aug 2026Verified 2 November 202681 published resultsOfficial source

SimpleQA Verified · Lumina Score

Epoch AI · 1,000 verified questions

LuminaBench
Show
20 of 75 models · best variant per model
03006009001,2001,143Gemini 4 ArgonHigh1,141GPT-6 AstraMax1,134GPT-6.1 SolMax1,127Claude Opus 5.5Max1,121Claude Fable 5.1Max1,116Gemini 3.8 FlashHigh1,114Gemini 3.7 FlashHigh1,104Gemini 3 FlashHigh1,102Gemini 3.6 FlashHigh1,102Gemini 3.5 FlashHigh1,088GPT-5.5xHigh1,078GPT-6 SolMax1,077Muse Spark 1.2xHigh1,066Muse Spark 1.11,064GPT-6.1 TerraMax1,061Muse Spark OpenxHigh1,059Grok 4.7xHigh1,058Qwen3.7-Max1,046Claude Opus 4.8Max1,046DeepSeek V4 Pro 0813Max

Epoch AI published accuracy and standard error; unspecified effort remains unknown. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Source & conversion

Epoch AI’s proportion-correct evaluations on 1,000 SimpleQA Verified questions with its current anti-abstention prompt.

Simple proportion of all questions answered correctly, displayed as a percentage; this is not F1 or correct-given-attempted. Publisher standard error is retained. No resource measurements are published.

How the Lumina conversion works

1,000 + 75 × ((original score − 42.01920621851017) / 17.861535099095793). The result is rounded once. Scores may tie.

Frozen initial eligible Epoch cohort. Each exact model release contributes one reference value: the arithmetic mean of all its retained observed configuration scores, so releases with more settings do not receive extra calibration weight. This is normalization only, not a combined index or an estimate. Subsequent source refreshes never refit this calibration.

Calibration
simpleqa-verified-z75-v1 · experimental
Reference mean
42.01920621851017
Population SD
17.861535099095793
Coverage
Standalone Reasoning benchmark.
Identity, exclusions & attribution

Exact model releases and explicit native effort settings are retained. Unknown effort remains unknown; labelled token budgets are a separate setting axis.

Dataset size is published; per-row valid sample counts, trial counts, serving API provider and unreported harness versions remain unknown.

12 historical records have unverified current-protocol provenance and remain in the retained original CSV.

Official benchmark · Data and attribution · CC BY 4.0

Publisher terms. Scores and resource figures are attributed to the original publisher.