LuminaBench
Scientific reasoning benchmark / OpenRouter

GPQA Diamond — OpenRouter

Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.

Accuracy

Version OpenRouter-gpqa_diamond-v1-2026-11-06Verified 6 November 20261185 published resultsOfficial source

GPQA Diamond — OpenRouter

LuminaBench
Show
20 of 145 models · best variant per model
03006009001,2001,090Gemini 4 Argon1,086Gemini 3.8 Flash1,085GPT-6 Astra Pro1,084Gemini 3.1 Pro Preview1,082Ling 3.1 Flash1,082GPT-6.1 Sol1,082GPT-6 Astra1,080Gemini 3.7 Flash1,079GPT-5.6 Sol Pro1,079Kimi K3.1 Pro1,079GPT-5.51,076Kimi K31,075Gemini 3.6 Flash1,075Claude Sonnet 5.51,074Grok 4.61,074DeepSeek V4.1 Flash 10171,073GPT-5.6 Sol1,072GLM 5.3 Flash1,072GPT-6 Sol1,072Gemini 3.5 Flash

Lumina Score vs Cost per Task

USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous.

LuminaBench
15 of 145 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.$0.003$0.01$0.03$0.1$0.36007008009001,0001,100Cost per task · USD (log scale)Lumina ScoreGemini 4 ArgonGemini 3.8 FlashGPT-6 Astra ProGemini 3.1 Pro PreviewLing 3.1 Flash

Lumina Score vs Tokens

USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous.

LuminaBench
15 of 145 families
3K10K30K6007008009001,0001,100Output tokens per task (log scale)Lumina ScoreGemini 4 ArgonGemini 3.8 FlashGPT-6 Astra ProGPT-6.1 SolLing 3.1 Flash

Lumina Score vs Time per Task

USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous.

LuminaBench
15 of 145 families
Lower resource use and higher score (green); higher resource use and lower score (red). Guides split at the visible selection’s medians.0.313106007008009001,0001,100Time per task · min (log scale)Lumina ScoreGemini 4 ArgonGemini 3.8 FlashGPT-6 Astra ProGPT-6 AstraClaude Sonnet 5.5Gemini 3.7 Flash

USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous. Missing metrics are omitted from their graph; quality scores remain available.

Results table

Method & coverage

Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.

USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous.

How the Lumina conversion works

1,000 + 75 × ((original score − 76.19983493872542) / 16.919627969627754). The result is rounded once. Scores may tie.

Frozen initial observed text-only cohort; one mean per exact publisher release/API, balancing repeated settings/routes within that release. Original score and protocol remain separate. No inferred variants or future refitting.

Calibration
openrouter-gpqa-diamond-z75-v1 · experimental
Reference mean
76.19983493872542
Population SD
16.919627969627754
Coverage
Standalone benchmark.
Identity, exclusions & attribution

Rows aggregate the benchmark-runs dataset exactly as the leaderboard pages do, over each benchmark's non-zero-score runs from the newest 90 days: accuracy is the task-weighted mean of per-run accuracy, accuracy_std_dev is the population standard deviation of per-run accuracy (null with a single run), and the per-task averages divide summed cost, generation time, and output tokens by summed tasks. Rows are ordered by accuracy descending, then model name.

Routing and search-lane observations are distinct configurations, not independent model releases. No effort is inferred from an unlabelled default or a turn budget.

Source standard deviation is retained in original evidence only, not presented as an inferred confidence interval.

Text questions and text function/search tools. HLE uses the explicitly documented text-only subset; multimodal questions excluded.

Standalone text benchmark; no new combined index. Source identity/effort/protocol qualifications remain in Data and CSV.

All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.

Official source · Publication terms / evidence

Publisher terms. Scores and resource figures are attributed to the original publisher.