LuminaBench
Coding benchmark / Aider

Aider Polyglot

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Two-attempt pass rate

Version Aider-Polyglot-225-20261106Verified 6 November 202669 published resultsOfficial source

Aider Polyglot

LuminaBench
Show
20 of 53 models · best variant per model
03006009001,2001,143GPT-5High1,133o3-proHigh1,127gemini-2.5-pro-preview-06-05 (32k think)1,122o3High1,116Grok 4High1,107Gemini 2.5 Pro Preview 05 061,099DeepSeek-V3.2-Exp (Reasoner)1,095gemini-2.5-pro-preview-03-251,092o4-miniHigh1,092Claude Opus 41,090DeepSeek-R1 (0528)1,086DeepSeek-V3.2-Exp (Chat)1,069claude-3-7-sonnet-20250219 (32k thinking tokens)1,066DeepSeek R1 + claude-3-5-sonnet-202410221,059o1-2024-12-17High1,058Claude Sonnet 41,055o3-miniHigh1,055claude-3-7-sonnet-20250219 (no thinking)1,052Qwen3 235B A22B diff, no think, Alibaba API1,051kimi-k2

Lumina Score vs Cost per Task

No comparable per-task resource measurements were published.

LuminaBench
15 of 36 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.$0.001$0.003$0.01$0.03$0.1$0.3$19001,0001,1001,200Cost per task · USD (log scale)Lumina ScoreGPT-5o3-proo3DeepSeek-V3.2-Exp (Reasoner)DeepSeek-V3.2-Exp (Chat)

Lumina Score vs Time per Task

No comparable per-task resource measurements were published.

LuminaBench
15 of 53 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.13109001,0001,1001,200Time per task · min (log scale)Lumina ScoreGPT-5o3-proClaude Opus 4gemini-2.5-pro-preview-03-25

No comparable per-task resource measurements were published. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Method & coverage

225 Exercism tasks across six languages. Published pass_rate_2 after feedback; preserve single/architect edit formats and Aider versions. Source total cost divided by declared task count; source mean seconds per case.

No comparable per-task resource measurements were published.

How the Lumina conversion works

1,000 + 75 × ((original score − 43.215895061728396) / 23.501882217576174). The result is rounded once. Scores may tie.

Frozen initial observed text-only cohort; one mean per exact publisher release/API, balancing repeated settings/routes within that release. Original score and protocol remain separate. No inferred variants or future refitting.

Calibration
aider-polyglot-z75-v1 · experimental
Reference mean
43.215895061728396
Population SD
23.501882217576174
Coverage
Standalone benchmark.
Identity, exclusions & attribution

Original measurements and exact source settings retained; unknown API, effort and harness versions remain unknown.

Text programming instructions, code editing and test feedback.

Standalone text benchmark; no new combined index. Source identity/effort/protocol qualifications remain in Data and CSV.

All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.

Official source · Publication terms / evidence

Publisher terms. Scores and resource figures are attributed to the original publisher.