HLE Search — OpenRouter
Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.Search score
HLE Search — OpenRouter
Lumina Score vs Cost per Task
USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous.
Lumina Score vs Time per Task
USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous.
USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous. Missing metrics are omitted from their graph; quality scores remain available.
Results table
Method & coverage
Runs pair a model with a web search engine through the OpenRouter server-tool or plugin surface. A run contributes only when it succeeded, is visible on openrouter.ai, has no epoch marker, ran through a public search surface, answered at least 45 questions, and is the latest run of its resume lineage; within a lane only runs sharing the most recent configuration are pooled. Score is pooled correct answers over pooled questions, which is the value the suite pages plot.
USD per task; time is mean generation time per task for GPQA/airline, mean search-task latency for search lanes. These definitions are not interchangeable. Positive published telemetry only; missing/zero telemetry is not represented as free or instantaneous.
How the Lumina conversion works
1,000 + 75 × ((original score − 68.41717138767086) / 1.8264290679540451). The result is rounded once. Scores may tie.
Frozen initial observed text-only cohort; one mean per exact publisher release/API, balancing repeated settings/routes within that release. Original score and protocol remain separate. No inferred variants or future refitting.
- Calibration
- openrouter-search-hle-z75-v1 · experimental
- Reference mean
- 68.41717138767086
- Population SD
- 1.8264290679540451
- Coverage
- Standalone benchmark.
Identity, exclusions & attribution
Runs pair a model with a web search engine through the OpenRouter server-tool or plugin surface. A run contributes only when it succeeded, is visible on openrouter.ai, has no epoch marker, ran through a public search surface, answered at least 45 questions, and is the latest run of its resume lineage; within a lane only runs sharing the most recent configuration are pooled. Score is pooled correct answers over pooled questions, which is the value the suite pages plot.
Routing and search-lane observations are distinct configurations, not independent model releases. No effort is inferred from an unlabelled default or a turn budget.
Source standard deviation is retained in original evidence only, not presented as an inferred confidence interval.
Text questions and text function/search tools. HLE uses the explicitly documented text-only subset; multimodal questions excluded.
Standalone text benchmark; no new combined index. Source identity/effort/protocol qualifications remain in Data and CSV.
All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.
Official source · Publication terms / evidence
Publisher terms. Scores and resource figures are attributed to the original publisher.