LuminaBench
Coding benchmark / Terminal-Bench / Harbor

Terminal-Bench 4.0

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Agentic problem solving across real terminal tasks. Published results, checked by Lumina.

Version 4.0Verified 30 October 202635 published resultsOfficial source
35 of 35 publisher records

Terminal-Bench · Lumina Score

Publisher results on a fixed Lumina scale. Experimental calibration.

LuminaBench
Show
20 of 35 configurations
03006009001,200Claude Opus 5.5 Max · Claude Code · 11091,109Claude Opus 5.5MaxClaude Sonnet 5.5 Max · Claude Code · 10951,095Claude Sonnet 5.5MaxGPT-6 Astra Max · Codex · 10791,079GPT-6 AstraMaxGPT-6.1 Sol Max · Codex · 10791,079GPT-6.1 SolMaxClaude Fable 5.1 Max · Claude Code · 10781,078Claude Fable 5.1MaxClaude Fable 5.1 xHigh · Claude Code · 10781,078Claude Fable 5.1xHighGPT-6 Astra High · Codex · 10781,078GPT-6 AstraHighGPT-6 Astra xHigh · Codex · 10781,078GPT-6 AstraxHighClaude Fable 5.1 High · Claude Code · 10631,063Claude Fable 5.1HighGPT-6 Astra Medium · Codex · 10621,062GPT-6 AstraMediumClaude Fable 5.1 Medium · Claude Code · 10601,060Claude Fable 5.1MediumClaude Opus 5 xHigh · Claude Code · 10601,060Claude Opus 5xHighClaude Opus 5 Max · Claude Code · 10511,051Claude Opus 5MaxGPT-6 Astra Low · Codex · 10461,046GPT-6 AstraLowClaude Opus 5 High · Claude Code · 10441,044Claude Opus 5HighGPT-6 Sol Max · Codex · 10401,040GPT-6 SolMaxClaude Opus 5 Medium · Claude Code · 10201,020Claude Opus 5MediumClaude Fable 5 Max · Claude Code · 10191,019Claude Fable 5MaxClaude Fable 5.1 Low · Claude Code · 10131,013Claude Fable 5.1LowGLM 5.3 Max · Claude Code · 10071,007GLM 5.3Max

Lumina Score vs Evaluation Cost

Publisher-reported evaluation total; rounded where shown. No per-task division.

LuminaBench
8 of 22 families
$1000$30009001,0001,100Evaluation cost · USD (publisher totals) (log scale)Lumina ScoreClaude Opus 5.5Claude Sonnet 5.5GPT-6 AstraGPT-6.1 SolClaude Fable 5.1Claude Opus 5GPT-6 SolClaude Fable 5

Lumina Score vs Evaluation Tokens

Publisher-reported total evaluation tokens. Rounded values retain their source precision.

LuminaBench
8 of 23 families
1B3B10B9001,0001,100Evaluation tokens · publisher totals (log scale)Lumina ScoreClaude Opus 5.5Claude Sonnet 5.5GPT-6 AstraGPT-6.1 SolClaude Fable 5.1Claude Opus 5GPT-6 SolClaude Fable 5

Publisher-reported resolution rates and 95% confidence intervals. Resource figures are evaluation totals; ≈ marks rounded source values. Incomplete cost coverage is retained on hover. Missing metrics are omitted from their graph; quality scores remain available.

All results

Terminal-Bench 4.0. Original publisher values and provisional Lumina conversion.
Rank
1Claude Opus 5.5Max1,10964.8%±3.1 pp$4,718.658,034,082,467Claude Code
2Claude Sonnet 5.5Max1,09561.8%±2.9 pp$7,333.6419,413,047,384Claude Code
3GPT-6 AstraMax1,07958.2%±2.8 pp≈$3.3k≈1.5BCodex
3GPT-6.1 SolMax1,07958.2%±3.1 pp$634.241,459,936,514Codex
5Claude Fable 5.1Max1,07857.9%±3.8 pp$6,243.52,746,221,560Claude Code
5Claude Fable 5.1xHigh1,07857.9%±3.4 pp≈$4.9k≈2.3BClaude Code
5GPT-6 AstraHigh1,07857.9%±3 pp≈$2.3k≈1.2BCodex
5GPT-6 AstraxHigh1,07857.9%±2.7 pp≈$2.4k≈1.2BCodex
9Claude Fable 5.1High1,06354.5%±3.4 pp≈$4.0k≈2.2BClaude Code
10GPT-6 AstraMedium1,06254.2%±2.7 pp≈$1.9k≈1.1BCodex
11Claude Fable 5.1Medium1,06053.9%±3.4 pp≈$2.8k≈1.6BClaude Code
11Claude Opus 5xHigh1,06053.9%±3.2 pp≈$6.1k≈6.9BClaude Code
13Claude Opus 5Max1,05151.8%±3.4 pp$5,969.116,527,147,637Claude Code
14GPT-6 AstraLow1,04650.6%±2.8 pp≈$1.6k≈0.9BCodex
15Claude Opus 5High1,04450.3%±3.7 pp≈$4.7k≈5.5BClaude Code
16GPT-6 SolMax1,04049.4%±3.2 pp$1,451.324,977,940,044Codex
17Claude Opus 5Medium1,02044.8%±3.8 pp≈$3.2k≈3.7BClaude Code
18Claude Fable 5Max1,01944.5%±3.8 pp$7,265.013,785,220,369Claude Code
19Claude Fable 5.1Low1,01343.3%±3.6 pp≈$2.4k≈1.3BClaude Code
20GLM 5.3Max1,00741.8%±3.2 pp$2,727.638,677,039,373Claude Code
21Grok 4.7xHigh98837.6%±3.5 pp$3,683.29324/330 trials5,485,862,389Grok Build
22GPT-5.6 SolMax98737.3%±3.8 pp$2,541.74,409,948,969Codex
23GLM 5.3 FlashNone98035.8%±3.5 pp$118.742,916,503,275Claude Code
24Claude Opus 5Low97634.8%±3.9 pp≈$2.4k≈2.7BClaude Code
25Qwen 3.8 Max 0902Max94127%±3.8 pp$4,635.672,957,967,711Claude Code
26Claude Opus 4.8Max92623.6%±3.6 pp$6,481.266,415,823,126Claude Code
27GPT-5.6 TerraMax91721.5%±3.3 pp$1,733.525,683,805,798Codex
28Grok 4.6High91220.3%±3.1 pp$3,591.584,001,923,428Grok Build
29Gemini 3.8 FlashHigh90619.1%±3.4 pp≈$1.8k17,189,729,495mini-SWE-agent
30GPT-5.6 LunaMax89917.3%±2.8 pp$346.6711,557,463,640Codex
31GPT-6 LunaMax89516.4%±2.7 pp$72.143,909,124,893Codex
32Muse Spark 1.3xHigh88614.5%±2.9 ppNot published9,010,923,518Muse Code
33Claude Sonnet 5Max87712.4%±3.1 pp$9,603.8621,562,180,742Claude Code
33Grok 4.5High87712.4%±2.6 pp≈$2.1k≈3.4BGrok Build
35Gemini 3.7 FlashHigh87211.2%±2.4 pp$1,261.8711,137,189,802mini-SWE-agent
1 · Claude Opus 5.5Max1109
Resolution rate
64.8%
95% CI
±3.1 pp
Evaluation cost · USD
$4,718.65
Evaluation tokens
8,034,082,467
Agent / harness
Claude Code
Evidence
Model profile ↗
2 · Claude Sonnet 5.5Max1095
Resolution rate
61.8%
95% CI
±2.9 pp
Evaluation cost · USD
$7,333.64
Evaluation tokens
19,413,047,384
Agent / harness
Claude Code
Evidence
Model profile ↗
3 · GPT-6 AstraMax1079
Resolution rate
58.2%
95% CI
±2.8 pp
Evaluation cost · USD
≈$3.3k
Evaluation tokens
≈1.5B
Agent / harness
Codex
Evidence
Model profile ↗
3 · GPT-6.1 SolMax1079
Resolution rate
58.2%
95% CI
±3.1 pp
Evaluation cost · USD
$634.24
Evaluation tokens
1,459,936,514
Agent / harness
Codex
Evidence
Model profile ↗
5 · Claude Fable 5.1Max1078
Resolution rate
57.9%
95% CI
±3.8 pp
Evaluation cost · USD
$6,243.5
Evaluation tokens
2,746,221,560
Agent / harness
Claude Code
Evidence
Model profile ↗
5 · Claude Fable 5.1xHigh1078
Resolution rate
57.9%
95% CI
±3.4 pp
Evaluation cost · USD
≈$4.9k
Evaluation tokens
≈2.3B
Agent / harness
Claude Code
Evidence
Model profile ↗
5 · GPT-6 AstraHigh1078
Resolution rate
57.9%
95% CI
±3 pp
Evaluation cost · USD
≈$2.3k
Evaluation tokens
≈1.2B
Agent / harness
Codex
Evidence
Model profile ↗
5 · GPT-6 AstraxHigh1078
Resolution rate
57.9%
95% CI
±2.7 pp
Evaluation cost · USD
≈$2.4k
Evaluation tokens
≈1.2B
Agent / harness
Codex
Evidence
Model profile ↗
9 · Claude Fable 5.1High1063
Resolution rate
54.5%
95% CI
±3.4 pp
Evaluation cost · USD
≈$4.0k
Evaluation tokens
≈2.2B
Agent / harness
Claude Code
Evidence
Model profile ↗
10 · GPT-6 AstraMedium1062
Resolution rate
54.2%
95% CI
±2.7 pp
Evaluation cost · USD
≈$1.9k
Evaluation tokens
≈1.1B
Agent / harness
Codex
Evidence
Model profile ↗
11 · Claude Fable 5.1Medium1060
Resolution rate
53.9%
95% CI
±3.4 pp
Evaluation cost · USD
≈$2.8k
Evaluation tokens
≈1.6B
Agent / harness
Claude Code
Evidence
Model profile ↗
11 · Claude Opus 5xHigh1060
Resolution rate
53.9%
95% CI
±3.2 pp
Evaluation cost · USD
≈$6.1k
Evaluation tokens
≈6.9B
Agent / harness
Claude Code
Evidence
Model profile ↗
13 · Claude Opus 5Max1051
Resolution rate
51.8%
95% CI
±3.4 pp
Evaluation cost · USD
$5,969.11
Evaluation tokens
6,527,147,637
Agent / harness
Claude Code
Evidence
Model profile ↗
14 · GPT-6 AstraLow1046
Resolution rate
50.6%
95% CI
±2.8 pp
Evaluation cost · USD
≈$1.6k
Evaluation tokens
≈0.9B
Agent / harness
Codex
Evidence
Model profile ↗
15 · Claude Opus 5High1044
Resolution rate
50.3%
95% CI
±3.7 pp
Evaluation cost · USD
≈$4.7k
Evaluation tokens
≈5.5B
Agent / harness
Claude Code
Evidence
Model profile ↗
16 · GPT-6 SolMax1040
Resolution rate
49.4%
95% CI
±3.2 pp
Evaluation cost · USD
$1,451.32
Evaluation tokens
4,977,940,044
Agent / harness
Codex
Evidence
Model profile ↗
17 · Claude Opus 5Medium1020
Resolution rate
44.8%
95% CI
±3.8 pp
Evaluation cost · USD
≈$3.2k
Evaluation tokens
≈3.7B
Agent / harness
Claude Code
Evidence
Model profile ↗
18 · Claude Fable 5Max1019
Resolution rate
44.5%
95% CI
±3.8 pp
Evaluation cost · USD
$7,265.01
Evaluation tokens
3,785,220,369
Agent / harness
Claude Code
Evidence
Model profile ↗
19 · Claude Fable 5.1Low1013
Resolution rate
43.3%
95% CI
±3.6 pp
Evaluation cost · USD
≈$2.4k
Evaluation tokens
≈1.3B
Agent / harness
Claude Code
Evidence
Model profile ↗
20 · GLM 5.3Max1007
Resolution rate
41.8%
95% CI
±3.2 pp
Evaluation cost · USD
$2,727.63
Evaluation tokens
8,677,039,373
Agent / harness
Claude Code
Evidence
Model profile ↗
21 · Grok 4.7xHigh988
Resolution rate
37.6%
95% CI
±3.5 pp
Evaluation cost · USD
$3,683.29
Evaluation tokens
5,485,862,389
Agent / harness
Grok Build
Evidence
Model profile ↗
22 · GPT-5.6 SolMax987
Resolution rate
37.3%
95% CI
±3.8 pp
Evaluation cost · USD
$2,541.7
Evaluation tokens
4,409,948,969
Agent / harness
Codex
Evidence
Model profile ↗
23 · GLM 5.3 FlashNone980
Resolution rate
35.8%
95% CI
±3.5 pp
Evaluation cost · USD
$118.74
Evaluation tokens
2,916,503,275
Agent / harness
Claude Code
Evidence
Model profile ↗
24 · Claude Opus 5Low976
Resolution rate
34.8%
95% CI
±3.9 pp
Evaluation cost · USD
≈$2.4k
Evaluation tokens
≈2.7B
Agent / harness
Claude Code
Evidence
Model profile ↗
25 · Qwen 3.8 Max 0902Max941
Resolution rate
27%
95% CI
±3.8 pp
Evaluation cost · USD
$4,635.67
Evaluation tokens
2,957,967,711
Agent / harness
Claude Code
Evidence
Model profile ↗
26 · Claude Opus 4.8Max926
Resolution rate
23.6%
95% CI
±3.6 pp
Evaluation cost · USD
$6,481.26
Evaluation tokens
6,415,823,126
Agent / harness
Claude Code
Evidence
Model profile ↗
27 · GPT-5.6 TerraMax917
Resolution rate
21.5%
95% CI
±3.3 pp
Evaluation cost · USD
$1,733.52
Evaluation tokens
5,683,805,798
Agent / harness
Codex
Evidence
Model profile ↗
28 · Grok 4.6High912
Resolution rate
20.3%
95% CI
±3.1 pp
Evaluation cost · USD
$3,591.58
Evaluation tokens
4,001,923,428
Agent / harness
Grok Build
Evidence
Model profile ↗
29 · Gemini 3.8 FlashHigh906
Resolution rate
19.1%
95% CI
±3.4 pp
Evaluation cost · USD
≈$1.8k
Evaluation tokens
17,189,729,495
Agent / harness
mini-SWE-agent
Evidence
Model profile ↗
30 · GPT-5.6 LunaMax899
Resolution rate
17.3%
95% CI
±2.8 pp
Evaluation cost · USD
$346.67
Evaluation tokens
11,557,463,640
Agent / harness
Codex
Evidence
Model profile ↗
31 · GPT-6 LunaMax895
Resolution rate
16.4%
95% CI
±2.7 pp
Evaluation cost · USD
$72.14
Evaluation tokens
3,909,124,893
Agent / harness
Codex
Evidence
Model profile ↗
32 · Muse Spark 1.3xHigh886
Resolution rate
14.5%
95% CI
±2.9 pp
Evaluation cost · USD
Not published
Evaluation tokens
9,010,923,518
Agent / harness
Muse Code
Evidence
Model profile ↗
33 · Claude Sonnet 5Max877
Resolution rate
12.4%
95% CI
±3.1 pp
Evaluation cost · USD
$9,603.86
Evaluation tokens
21,562,180,742
Agent / harness
Claude Code
Evidence
Model profile ↗
33 · Grok 4.5High877
Resolution rate
12.4%
95% CI
±2.6 pp
Evaluation cost · USD
≈$2.1k
Evaluation tokens
≈3.4B
Agent / harness
Grok Build
Evidence
Model profile ↗
35 · Gemini 3.7 FlashHigh872
Resolution rate
11.2%
95% CI
±2.4 pp
Evaluation cost · USD
$1,261.87
Evaluation tokens
11,137,189,802
Agent / harness
mini-SWE-agent
Evidence
Model profile ↗

Source & conversion

Terminal-Bench 4.0 measures agentic problem solving across 66 terminal tasks. The original resolution rates and publisher-reported 95% confidence intervals are retained.

Cost and tokens are evaluation totals across trials, not per-task averages. Exact repository or linked public-job totals are used only for matching published rows; other values retain their rounded source precision. Grok 4.7 cost covers 324/330 trials. Missing cost remains unavailable. No per-trial division or aggregate resource totals are calculated.

How the Lumina conversion works

1,000 + 75 × ((original score − 40.251428571428555) / 16.964052269070475). The result is rounded once. Scores may tie.

Frozen initial 35 publisher rows across 23 exact model families, all published reasoning settings including Claude Max. Variant/harness-weighted population; no recalibration when new rows appear.

Calibration
terminal-bench-4-z75-v1 · experimental
Reference mean
40.251428571428555
Population SD
16.964052269070475
Coverage
Independent benchmark; no contribution to the current Capabilities Index.
Identity, exclusions & attribution

The coding combination matches curator-verified exact release labels and reasoning settings while keeping source-specific harnesses. Unknown API snapshots remain unknown.

Current Grok 4.5/4.6 High labels conflict with repository None metadata. Those source records remain visible and receive separate provisional evidence until their underlying identities are reviewed.

Publisher resources can be rounded or incomplete; they do not measure the cost of the Coding Index.

All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.

Official leaderboard and export · Official dataset and methodology · Published submission schema · Repository licence

Publisher terms. Scores and resource figures are attributed to the original publisher.