ForecastBench
Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.Overall Brier Index
ForecastBench
No comparable per-task resource measurements were published. Missing metrics are omitted from their graph; quality scores remain available.
Results table
Method & coverage
Original baseline leaderboard Overall Brier Index, higher is better: difficulty-adjusted Brier scores aggregate across categories before nonlinear conversion. Do not average the two displayed category indexes or use unadjusted Brier error. AI ensembles remain labelled ensembles; human/statistical references excluded.
No comparable per-task resource measurements were published.
How the Lumina conversion works
1,000 + 75 × ((original score − 57.04672131147541) / 3.062469388973263). The result is rounded once. Scores may tie.
Frozen initial observed text-only cohort; one mean per exact publisher release/API, balancing repeated settings/routes within that release. Original score and protocol remain separate. No inferred variants or future refitting.
- Calibration
- forecastbench-z75-v1 · experimental
- Reference mean
- 57.04672131147541
- Population SD
- 3.062469388973263
- Coverage
- Standalone benchmark.
Identity, exclusions & attribution
Higher Overall Brier Index is better. Underlying raw Brier errors are retained as provenance, not substituted as the ranking metric.
This source data and its Lumina-normalized adaptation are CC BY-SA 4.0; attribution and source link retained. No licence is imposed on unrelated benchmark data.
Forecast horizons are task characteristics, not inference time. No cost, generation time or tokens invented.
Text probabilistic forecasting of questions; no image/audio inputs.
Standalone text benchmark; no new combined index. Source identity/effort/protocol qualifications remain in Data and CSV.
All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.
Official source · Publication terms / evidence
Publisher terms. Scores and resource figures are attributed to the original publisher.