LuminaBench
Reasoning benchmark / Scale AI / CAIS

HLE-Diamond

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Overall accuracy on the 1,000-question HLE-Diamond evaluation.

Version Diamond-1000Verified 31 October 202619 published resultsOfficial source

HLE-Diamond · Lumina Score

Overall accuracy · 1,000 questions

LuminaBench
Show
19 of 19 configurations
03006009001,2001,149GPT-6 Astra1,118Claude Opus 5.51,109Gemini 4 Argon1,098Claude Fable 5.11,030Claude Opus 51,024Claude Sonnet 5.51,023Muse Wonder1,014GPT-6.1 Terra1,006Gemini 3.8 Flash1,004GPT-6 Sol990GPT-5.6 Sol972Muse Spark Open958Muse Spark 1.3948Grok 4.7941Kimi K3932DeepSeek V4.1 Flash 1017910GLM 5.3893DeepSeek V4 Pro882Inkling

Lumina Score vs Reasoning Accuracy

Reasoning partition · 500 questions

LuminaBench
7 of 7 families
30405060709001,0001,1001,200Reasoning accuracy · %Lumina ScoreGPT-6 AstraMuse Spark 1.3Grok 4.7Claude Opus 5.5Claude Fable 5.1GPT-6 SolGemini 3.8 Flash

Lumina Score vs Knowledge Accuracy

Knowledge partition · 500 questions

LuminaBench
7 of 7 families
203040509001,0001,1001,200Knowledge accuracy · %Lumina ScoreGrok 4.7GPT-6 AstraClaude Opus 5.5Claude Fable 5.1Muse Spark 1.3GPT-6 SolGemini 3.8 Flash

Lumina Score vs Calibration Error

Publisher-reported RMS calibration error; lower is better.

LuminaBench
15 of 18 families
2040609001,0001,1001,200RMS calibration errorLumina ScoreGPT-6 AstraClaude Opus 5.5Gemini 3.8 FlashGemini 4 ArgonKimi K3Grok 4.7Claude Fable 5.1Muse Spark 1.3Muse Spark OpenGPT-5.6 SolGPT-6 SolGPT-6.1 TerraClaude Opus 5Claude Sonnet 5.5

Publisher uncertainty and available partition scores are retained. Unspecified reasoning effort remains unknown. Missing metrics are omitted from their graph; quality scores remain available.

Results table

HLE-Diamond Diamond-1000. Original publisher values and provisional Lumina conversion.
Rank
1GPT-6 AstraNot specified1,14960.6%±3 pp75.645.628.1
2Claude Opus 5.5Not specified1,11855%±3.1 pp63.246.87.1
3Gemini 4 ArgonNot specified1,10953.2%±3.07 ppNot publishedNot published53.3
4Claude Fable 5.1Not specified1,09851.3%±3.1 pp6240.613.8
5Claude Opus 5Not specified1,03038.6%±3 ppNot publishedNot published21.5
6Claude Sonnet 5.5Not specified1,02437.5%±3 ppNot publishedNot published36.8
7Muse WonderNot specified1,02337.3%±3.04 ppNot publishedNot published31.1
8GPT-6.1 TerraNot specified1,01435.7%±3 ppNot publishedNot published32.9
9Gemini 3.8 FlashNot specified1,00634.3%±2.9 pp38.63057.2
10GPT-6 SolNot specified1,00433.8%±2.9 pp44.223.438.8
11GPT-5.6 SolNot specified99031.2%±2.9 ppNot publishedNot published53.8
12Muse Spark OpenNot specified97228%±2.86 ppNot publishedNot published32.5
13Muse Spark 1.3Not specified95825.4%±2.7 pp31.619.248.2
14Grok 4.7Not specified94823.4%±2.6 pp32.414.428.9
15Kimi K3Not specified94122.2%±2.6 ppNot publishedNot published54.4
16DeepSeek V4.1 Flash 1017Not specified93220.5%±2.5 ppNot publishedNot publishedNot published
17GLM 5.3Not specified91016.4%±2.3 ppNot publishedNot published53.1
18DeepSeek V4 ProNot specified89313.4%±2.1 ppNot publishedNot published76.3
19InklingNot specified88211.2%±2.11 ppNot publishedNot published64.6
1 · GPT-6 AstraNot specified1149
Accuracy
60.6%
Published ±
±3 pp
Reasoning · %
75.6
Knowledge · %
45.6
Calibration error
28.1
Evidence
Model profile ↗
2 · Claude Opus 5.5Not specified1118
Accuracy
55%
Published ±
±3.1 pp
Reasoning · %
63.2
Knowledge · %
46.8
Calibration error
7.1
Evidence
Model profile ↗
3 · Gemini 4 ArgonNot specified1109
Accuracy
53.2%
Published ±
±3.07 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
53.3
Evidence
Model profile ↗
4 · Claude Fable 5.1Not specified1098
Accuracy
51.3%
Published ±
±3.1 pp
Reasoning · %
62
Knowledge · %
40.6
Calibration error
13.8
Evidence
Model profile ↗
5 · Claude Opus 5Not specified1030
Accuracy
38.6%
Published ±
±3 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
21.5
Evidence
Model profile ↗
6 · Claude Sonnet 5.5Not specified1024
Accuracy
37.5%
Published ±
±3 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
36.8
Evidence
Model profile ↗
7 · Muse WonderNot specified1023
Accuracy
37.3%
Published ±
±3.04 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
31.1
Evidence
Model profile ↗
8 · GPT-6.1 TerraNot specified1014
Accuracy
35.7%
Published ±
±3 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
32.9
Evidence
Model profile ↗
9 · Gemini 3.8 FlashNot specified1006
Accuracy
34.3%
Published ±
±2.9 pp
Reasoning · %
38.6
Knowledge · %
30
Calibration error
57.2
Evidence
Model profile ↗
10 · GPT-6 SolNot specified1004
Accuracy
33.8%
Published ±
±2.9 pp
Reasoning · %
44.2
Knowledge · %
23.4
Calibration error
38.8
Evidence
Model profile ↗
11 · GPT-5.6 SolNot specified990
Accuracy
31.2%
Published ±
±2.9 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
53.8
Evidence
Model profile ↗
12 · Muse Spark OpenNot specified972
Accuracy
28%
Published ±
±2.86 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
32.5
Evidence
Model profile ↗
13 · Muse Spark 1.3Not specified958
Accuracy
25.4%
Published ±
±2.7 pp
Reasoning · %
31.6
Knowledge · %
19.2
Calibration error
48.2
Evidence
Model profile ↗
14 · Grok 4.7Not specified948
Accuracy
23.4%
Published ±
±2.6 pp
Reasoning · %
32.4
Knowledge · %
14.4
Calibration error
28.9
Evidence
Model profile ↗
15 · Kimi K3Not specified941
Accuracy
22.2%
Published ±
±2.6 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
54.4
Evidence
Model profile ↗
16 · DeepSeek V4.1 Flash 1017Not specified932
Accuracy
20.5%
Published ±
±2.5 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
Not published
Evidence
Model profile ↗
17 · GLM 5.3Not specified910
Accuracy
16.4%
Published ±
±2.3 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
53.1
Evidence
Model profile ↗
18 · DeepSeek V4 ProNot specified893
Accuracy
13.4%
Published ±
±2.1 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
76.3
Evidence
Model profile ↗
19 · InklingNot specified882
Accuracy
11.2%
Published ±
±2.11 pp
Reasoning · %
Not published
Knowledge · %
Not published
Calibration error
64.6
Evidence
Model profile ↗

Source & conversion

Overall accuracy across 1,000 questions. Reasoning and knowledge partitions are supporting measurements, not additional index components.

Uncertainty is retained exactly as the publisher reports it; the page does not identify a confidence level or reasoning effort. Partition scores are available only for seven published rows. Missing values remain unknown.

How the Lumina conversion works

1,000 + 75 × ((original score − 33.10526315789474) / 13.868194345495455). The result is rounded once. Scores may tie.

Frozen initial verified cohort of 19 configurations; Each published configuration has one reference observation; no future automatic refit.

Calibration
hle-diamond-z75-v1 · experimental
Reference mean
33.10526315789474
Population SD
13.868194345495455
Coverage
Standalone Reasoning benchmark.
Identity, exclusions & attribution

Verified publisher data, not an independent Lumina rerun.

Official leaderboard · Benchmark description

Publisher terms. Scores and resource figures are attributed to the original publisher.