Forcebench — Full API
Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.Salesforce engineering across 272 tasks and 15 suites.
Forcebench — Full API · Lumina Score
Lumina Score vs Cost per Task
Cumulative c@3 attempts at the dated API route price
Lumina Score vs Output Tokens per Task
Service-counted output per task across c@3 attempts
Lumina Score vs Time per Task
Median cumulative task time across c@3 attempts; service and concurrency affect latency
Full hosted/API evaluation, c@3: tasks solved within three feedback-assisted attempts, equal weight per suite. Pass@1 is separate. Local runs, lite sweeps and missing retries do not enter this ranking. Publisher 95% intervals resample tasks within suites; attempt ceilings are not actual attempt counts. Missing metrics are omitted from their graph; quality scores remain available.
Results table
Method & coverage
Full hosted/API evaluation, c@3: tasks solved within three feedback-assisted attempts, equal weight per suite. Pass@1 is separate. Local runs, lite sweeps and missing retries do not enter this ranking. Publisher 95% intervals resample tasks within suites; attempt ceilings are not actual attempt counts.
Publisher c@3 cumulative-attempt resources per task: dated route-list-price USD, service-reported mean output tokens, median wall time across tasks. Missing prices stay null. No cost-per-solved substitution or pass@1 resources.
How the Lumina conversion works
1,000 + 75 × ((original score − 71.8729885057471) / 20.18635763824424). The result is rounded once. Scores may tie.
Frozen initial eligible publisher cohort; one arithmetic mean per exact release/system, equally weighted across releases. Subsequent refreshes never refit. Strictly increasing conversion preserves source ordering at full precision; rounded display scores may tie.
- Calibration
- forcebench-api-z75-v1 · experimental
- Reference mean
- 71.8729885057471
- Population SD
- 20.18635763824424
- Coverage
- Standalone benchmark.
Identity, exclusions & attribution
Exact source release, effort, route and harness retained. Unknown evaluation dates stay unknown; no cross-source score estimates or combined indexes.
Code licensing is not assumed to clear separately published result redistribution.
All published reasoning settings, including Claude Max, are included by default. Display labels use Low and xHigh; source labels remain in the retained records.
Publisher terms. Scores and resource figures are attributed to the original publisher.