LuminaBench
Coding benchmark / SWE-bench authors

SWE-bench Verified — Original

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Resolved

Version SWE-bench-Verified-193160a463a4Verified 6 November 2026180 published resultsOfficial source

SWE-bench Verified — Original

LuminaBench
Show
20 of 83 models · best variant per model
03006009001,2001,109Claude 4.5 Opus1,108Doubao-Seed-Code1,103Gemini 3 Pro Preview1,100Claude 4 Sonnet1,100Multiple1,097Gemini 3 FlashHigh1,097MiniMax M2.5High1,096Claude 4.6 Opus1,093Claude Sonnet 41,093Claude 4.5 Sonnet1,092GPT-51,088Claude 4 Opus1,086GLM-5High1,086GPT-5.2High1,086GPT-5.2-Codex1,081kimi-k21,079Kimi K2.5High1,078Undisclosed1,076DeepSeek V3.2High1,075Qwen3-Coder-480B-A35B-Instruct

Lumina Score vs Cost per Task

No comparable per-task resource measurements were published.

LuminaBench
15 of 39 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.$0.1$0.3$18009001,0001,100Cost per task · USD (log scale)Lumina ScoreMiniMax M2.5Claude 4.5 OpusGemini 3 FlashGLM-5

No comparable per-task resource measurements were published. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Method & coverage

Original Verified submission leaderboard; exact complete model+agent systems, checked flags and warnings preserved. No multimodal results imported; submissions are not interchangeable native-model measurements.

No comparable per-task resource measurements were published.

How the Lumina conversion works

1,000 + 75 × ((original score − 48.57934937253757) / 21.061046934777757). The result is rounded once. Scores may tie.

Frozen initial observed text-only cohort; one mean per exact publisher release/API, balancing repeated settings/routes within that release. Original score and protocol remain separate. No inferred variants or future refitting.

Calibration
swe-bench-verified-original-z75-v1 · experimental
Reference mean
48.57934937253757
Population SD
21.061046934777757
Coverage
Standalone benchmark.
Identity, exclusions & attribution

Original measurements and exact source settings retained; unknown API, effort and harness versions remain unknown.

Text software issue descriptions and code/terminal agent tools; Multimodal board excluded.

Standalone text benchmark; no new combined index. Source identity/effort/protocol qualifications remain in Data and CSV.

All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.

Official source

Publisher terms. Scores and resource figures are attributed to the original publisher.