LuminaBench
Coding benchmark / htihle

WeirdML v3

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Machine-learning implementation across 11 tasks and 15 configurations.

Version v3Verified 2 November 202627 published resultsOfficial source

WeirdML v3 · Lumina Score

LuminaBench
Show
20 of 27 models · best variant per model
03006009001,2001,197GPT-6 AstraxHigh1,168GPT-6.1 SolxHigh1,119Claude Opus 5.5xHigh1,097GPT-6.1 TerraxHigh1,082Claude Fable 5.1xHigh1,041Gemini 4 ArgonHigh1,039Claude Sonnet 5.5xHigh1,037GPT-6 SolxHigh1,032Claude Opus 5xHigh1,008Claude Fable 5xHigh1,005Muse WonderxHigh1,004GPT-5.6 SolxHigh981Claude Haiku 5.5xHigh963DeepSeek V4.1 Flash 1017High958Grok 4.7xHigh954GPT-6 LunaxHigh951Kimi K3High950Gemini 3.8 FlashHigh945Muse Spark 1.3xHigh944Claude Opus 4.5High

Lumina Score vs Cost per Run

Publisher-reported means at the selected protocol

LuminaBench
15 of 27 families
$1$3$10$309001,0001,1001,200Mean API cost/run · USD (log scale)Lumina ScoreDeepSeek V4.1 Flash 1017GPT-6 AstraGrok 4.7Claude Fable 5Claude Haiku 5.5GPT-6.1 SolGPT-5.6 SolMuse WonderClaude Opus 5.5Claude Opus 5GPT-6 SolClaude Sonnet 5.5Gemini 4 ArgonGPT-6.1 TerraClaude Fable 5.1

Lumina Score vs Output Tokens per Run

Measured output tokens; execution budgets are separate

LuminaBench
15 of 27 families
101K310K952K9001,0001,1001,200Mean output tokens/run (log scale)Lumina ScoreGPT-5.6 SolGPT-6 AstraGrok 4.7DeepSeek V4.1 Flash 1017Claude Haiku 5.5GPT-6.1 SolMuse WonderClaude Fable 5GPT-6 SolGPT-6.1 TerraClaude Opus 5.5Claude Opus 5Claude Sonnet 5.5Gemini 4 ArgonClaude Fable 5.1

Publisher effective score: 80% normalized log-token AUC and 20% final value, equally weighted across 11 tasks. Hint twins share task weight. Approximate 95% run-uncertainty bounds assume fixed tasks, independent runs and pooled variance. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Method & coverage

Publisher effective score: 80% normalized log-token AUC and 20% final value, equally weighted across 11 tasks. Hint twins share task weight. Approximate 95% run-uncertainty bounds assume fixed tasks, independent runs and pooled variance.

Publisher mean API cost and output tokens per run; equivalent budget tokens are not output telemetry.

How the Lumina conversion works

1,000 + 75 × ((original score − 0.14436087124218375) / 0.10603183773408377). The result is rounded once. Scores may tie.

Frozen initial primary-protocol cohort; one mean per exact release/system, equally weighted. ALE reference uses self-refine ×1 only. Subsequent refreshes never refit. Strictly increasing conversion preserves source ordering at full precision; rounded display scores may tie.

Calibration
weirdml-v3-z75-v1 · experimental
Reference mean
0.14436087124218375
Population SD
0.10603183773408377
Coverage
Standalone benchmark.
Identity, exclusions & attribution

Exact source release, effort, route and harness retained. Unknown evaluation dates stay unknown; no cross-source score estimates or combined indexes.

Code licensing is not assumed to clear separately published result redistribution.

All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.

Official results

Publisher terms. Scores and resource figures are attributed to the original publisher.