OTIS Mock AIME 2024–2025
Lumina VerifiedLumina checked the published sources, versions, model labels, score extraction and conversion. External evaluations were run by their publishers; Lumina has not independently rerun or reproduced them.Mathematical reasoning on 45 mock AIME problems from the 2024–2025 question set.
OTIS Mock AIME 2024–2025 · Lumina Score
Epoch AI · 45 mock AIME questions
Epoch AI published accuracy and standard error; unspecified effort remains unknown. Missing metrics are omitted from their graph; quality scores remain available.
Results table
Source & conversion
Epoch AI’s maintained evaluations on the 45-question OTIS Mock AIME 2024–2025 set.
Proportion of questions answered correctly, displayed as a percentage. Genuine ties and 100% values stay unchanged. One current Gemini 2.5 Pro row has different mean and selected best-across-scorer values; the export does not identify which score its stderr describes, so it is preserved as unassociated publisher uncertainty.
How the Lumina conversion works
1,000 + 75 × ((original score − 68.81394446347683) / 31.64713712096185). The result is rounded once. Scores may tie.
Frozen initial eligible Epoch cohort. Each exact model release contributes one reference value: the arithmetic mean of all its retained observed configuration scores, so releases with more settings do not receive extra calibration weight. This is normalization only, not a combined index or an estimate. Subsequent source refreshes never refit this calibration.
- Calibration
- otis-mock-aime-z75-v1 · experimental
- Reference mean
- 68.81394446347683
- Population SD
- 31.64713712096185
- Coverage
- Standalone Reasoning benchmark.
Identity, exclusions & attribution
Exact model releases and explicit native effort settings are retained. Unknown effort remains unknown; labelled token budgets are a separate setting axis.
Dataset size is published; per-row valid sample counts, trial counts, serving API provider and unreported harness versions remain unknown.
87 historical records have unverified current-protocol provenance and remain in the retained original CSV.
Official benchmark · Data and attribution · CC BY 4.0
Publisher terms. Scores and resource figures are attributed to the original publisher.