LuminaBench
Reasoning benchmark / Epoch AI

Chess Puzzles

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Text-based chess puzzles, selecting the best next move from a FEN position.

Version 2Verified 2 November 2026154 published resultsOfficial source

Chess Puzzles · Lumina Score

Epoch AI · v2 · 100 text chess positions

LuminaBench
Show
20 of 83 models · best variant per model
03006009001,2001,191GPT-6 AstraProMax1,158GPT-6.1 SolMax1,158GPT-5.6 SolProMax1,153Claude Opus 5.5Max1,149Gemini 3.7 FlashHigh1,137GPT-5.5xHigh1,132GPT-5.5 ProxHigh1,124GPT-5.2 ProxHigh1,120GPT-5.4 ProxHigh1,120Gemini 4 ArgonHigh1,120Claude Sonnet 5.5Max1,116Gemini 3.8 FlashHigh1,107GPT-6 SolMax1,103Claude Haiku 5.5Max1,091Claude Opus 4.8Max1,087Gemini 3.1 ProHigh1,074GPT-5.4xHigh1,074DeepSeek V4.1 FlashMax1,074Claude Fable 5.1Max1,074GPT-5 ProHigh

Epoch AI published accuracy and standard error; unspecified effort remains unknown. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Source & conversion

Epoch AI’s current v2 evaluations on 100 chess positions presented as FEN text.

Proportion of correctly solved FEN positions, displayed as a percentage. Exported standard errors are retained; no resource measurements are published.

How the Lumina conversion works

1,000 + 75 × ((original score − 30.29759036144578) / 17.949364323060077). The result is rounded once. Scores may tie.

Frozen initial eligible Epoch cohort. Each exact model release contributes one reference value: the arithmetic mean of all its retained observed configuration scores, so releases with more settings do not receive extra calibration weight. This is normalization only, not a combined index or an estimate. Subsequent source refreshes never refit this calibration.

Calibration
chess-puzzles-z75-v1 · experimental
Reference mean
30.29759036144578
Population SD
17.949364323060077
Coverage
Standalone Reasoning benchmark.
Identity, exclusions & attribution

Exact model releases and explicit native effort settings are retained. Unknown effort remains unknown; labelled token budgets are a separate setting axis.

Dataset size is published; per-row valid sample counts, trial counts, serving API provider and unreported harness versions remain unknown.

0 historical records have unverified current-protocol provenance and remain in the retained original CSV.

Official benchmark · Data and attribution · CC BY 4.0

Publisher terms. Scores and resource figures are attributed to the original publisher.