American Invitational Mathematics Examination 2024

Accessible data for American Invitational Mathematics Examination 2024 leaderboard
Model	Rank	Provider	Score
GPT-5	#1	OpenAI	95.7%
Grok 4	#2	xAI	94.3%
o4-mini	#3	OpenAI	94%
Qwen3 235B A22B 2507 (Reasoning)	#4	Alibaba Cloud	94%
Grok 3 mini Reasoning (high)	#5	SpaceXAI	93.3%
GPT-5 (medium)	#6	OpenAI	91.7%
Qwen3 30B A3B 2507 (Reasoning)	#7	Alibaba Cloud	90.7%
o3	#8	OpenAI	90.3%
DeepSeek-R1	#9	DeepSeek	89.3%
Gemini 2.5 Pro	#10	Google	88.7%

Model

Rank

Provider

Score

GPT-5

OpenAI

95.7%

Grok 4

xAI

94.3%

o4-mini

OpenAI

94%

Qwen3 235B A22B 2507 (Reasoning)

Alibaba Cloud

94%

Grok 3 mini Reasoning (high)

SpaceXAI

93.3%

GPT-5 (medium)

OpenAI

91.7%

Qwen3 30B A3B 2507 (Reasoning)

Alibaba Cloud

90.7%

OpenAI

90.3%

DeepSeek-R1

DeepSeek

89.3%

Gemini 2.5 Pro

#10

Google

88.7%

American Invitational Mathematics Examination 2024 leaderboard ranked by score
Rank	Model	Provider	License	Evidence use	Score
#1	GPT-5 gpt-5	OpenAI	closed	Reference only	95.7%
#2	Grok 4 grok-4	xAI	closed	Reference only	94.3%
#3	o4-mini o4-mini	OpenAI	closed	Reference only	94%
#4	Qwen3 235B A22B 2507 (Reasoning) qwen3-235b-a22b-instruct-2507-reasoning	Alibaba Cloud	—	Reference only	94%
#5	Grok 3 mini Reasoning (high) grok-3-mini-reasoning	xAI	—	Reference only	93.3%
#6	GPT-5 (medium) gpt-5-medium	OpenAI	closed	Reference only	91.7%
#7	Qwen3 30B A3B 2507 (Reasoning) qwen3-30b-a3b-2507-reasoning	Alibaba Cloud	—	Reference only	90.7%
#8	o3 o3	OpenAI	closed	Reference only	90.3%
#9	DeepSeek-R1 deepseek-r1	DeepSeek	open	Reference only	89.3%
#10	Gemini 2.5 Pro gemini-2.5-pro	Google	closed	Reference only	88.7%
#11	GLM-4.5 (Reasoning) glm-4.5	Z.AI	open	Reference only	87.3%
#12	Gemini 2.5 Pro Preview (Mar' 25) gemini-2-5-pro-03-25	Google	—	Reference only	87%
#13	Llama Nemotron Super 49B v1.5 (Reasoning) llama-nemotron-super-49b-v1-5-reasoning	NVIDIA	—	Reference only	86%
#14	o3-mini (high) o3-mini-high	OpenAI	—	Reference only	86%
#15	MiniMax M1 80k minimax-m1-80k	MiniMax	closed	Reference only	84.7%
#16	EXAONE 4.0 32B (Reasoning) exaone-4-0-32b-reasoning	LG AI Research	—	Reference only	84.3%
#17	Gemini 2.5 Flash Preview (Reasoning) gemini-2-5-flash-reasoning-04-2025	Google	—	Reference only	84.3%
#18	Gemini 2.5 Pro Preview (May' 25) gemini-2-5-pro-05-06	Google	—	Reference only	84.3%
#19	Qwen3 235B A22B (Reasoning) qwen3-235b-a22b-instruct-reasoning	Alibaba Cloud	—	Reference only	84%
#20	GPT-5 (low) gpt-5-low	OpenAI	—	Reference only	83%
#21	Gemini 2.5 Flash (Reasoning) gemini-2-5-flash-reasoning	Google	—	Reference only	82.3%
#22	MiniMax M1 40k minimax-m1-40k	MiniMax	—	Reference only	81.3%
#23	Qwen3 32B (Reasoning) qwen3-32b-instruct-reasoning	Alibaba Cloud	—	Reference only	80.7%
#24	Sonar Reasoning Pro sonar-reasoning-pro	Perplexity	closed	Reference only	79%
#25	QwQ 32B qwq-32b	Alibaba Cloud	—	Reference only	78%
#26	Claude 4 Sonnet (Reasoning) claude-4-sonnet-thinking	Anthropic	—	Reference only	77.3%
#27	o3-mini o3-mini	OpenAI	closed	Reference only	77%
#28	Sonar Reasoning sonar-reasoning	Perplexity	—	Reference only	77%
#29	Qwen3 14B (Reasoning) qwen3-14b-instruct-reasoning	Alibaba Cloud	—	Reference only	76.3%
#30	Claude 4 Opus (Reasoning) claude-4-opus-thinking	Anthropic	—	Reference only	75.7%
#31	Qwen3 30B A3B (Reasoning) qwen3-30b-a3b-instruct-reasoning	Alibaba Cloud	—	Reference only	75.3%
#32	Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) llama-3-1-nemotron-ultra-253b-v1-reasoning	NVIDIA	—	Reference only	74.7%
#33	Qwen3 8B (Reasoning) qwen3-8b-instruct-reasoning	Alibaba Cloud	—	Reference only	74.7%
#34	Qwen3 30B A3B 2507 Instruct qwen3-30b-a3b-2507	Alibaba Cloud	—	Reference only	72.7%
#35	o1 o1	OpenAI	closed	Reference only	72.3%
#36	Qwen3 235B A22B 2507 Instruct qwen3-235b-a22b-instruct-2507	Alibaba Cloud	—	Reference only	71.7%
#37	Magistral Small 1 magistral-small	Mistral AI	open	Reference only	71.3%
#38	Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) llama-3-1-nemotron-nano-4b-reasoning	NVIDIA	—	Reference only	70.7%
#39	Gemini 2.5 Flash-Lite (Reasoning) gemini-2-5-flash-lite-reasoning	Google	—	Reference only	70.3%
#40	Magistral Medium 1 magistral-medium	Mistral AI	—	Reference only	70%
#41	Kimi K2 kimi-k2	Kimi	—	Reference only	69.3%
#42	Solar Pro 2 (Reasoning) solar-pro-2-reasoning	Upstage	—	Reference only	69%
#43	DeepSeek R1 Distill Qwen 32B deepseek-r1-distill-qwen-32b	DeepSeek	open	Reference only	68.7%
#44	DeepSeek R1 (Jan '25) deepseek-r1-0120	DeepSeek	—	Reference only	68.3%
#45	GLM-4.5-Air glm-4-5-air	Z.AI	closed	Reference only	67.3%
#46	DeepSeek R1 Distill Llama 70B deepseek-r1-distill-llama-70b	DeepSeek	—	Reference only	67%
#47	DeepSeek R1 Distill Qwen 14B deepseek-r1-distill-qwen-14b	DeepSeek	—	Reference only	66.7%
#48	Solar Pro 2 (Preview) (Reasoning) solar-pro-2-preview-reasoning	Upstage	—	Reference only	66.3%
#49	Qwen3 4B (Reasoning) qwen3-4b-instruct-reasoning	Alibaba Cloud	—	Reference only	65.7%
#50	DeepSeek R1 0528 Qwen3 8B deepseek-r1-qwen3-8b	DeepSeek	—	Reference only	65%
#51	o1-mini o1-mini	OpenAI	—	Reference only	60.3%
#52	Llama 3.3 Nemotron Super 49B v1 (Reasoning) llama-3-3-nemotron-super-49b-reasoning	NVIDIA	—	Reference only	58.3%
#53	Claude 4 Opus (Non-reasoning) claude-4-opus	Anthropic	—	Reference only	56.3%
#54	DeepSeek V3 0324 deepseek-v3-0324	DeepSeek	—	Reference only	52%
#55	Qwen3 1.7B (Reasoning) qwen3-1.7b-instruct-reasoning	Alibaba Cloud	—	Reference only	51%
#56	Reka Flash 3 reka-flash-3	Reka AI	—	Reference only	51%
#57	Gemini 2.0 Flash Thinking Experimental (Jan '25) gemini-2-0-flash-thinking-exp-0121	Google	—	Reference only	50%
#58	Gemini 2.5 Flash gemini-2.5-flash	Google	closed	Reference only	50%
#59	Gemini 2.5 Flash-Lite (Non-reasoning) gemini-2-5-flash-lite	Google	—	Reference only	50%
#60	ERNIE 4.5 300B A47B ernie-4-5-300b-a47b	Baidu	—	Reference only	49.3%
#61	Claude 3.7 Sonnet (Reasoning) claude-3-7-sonnet-thinking	Anthropic	—	Reference only	48.7%
#62	Sonar sonar	Perplexity	closed	Reference only	48.7%
#63	Qwen3 Coder 480B A35B Instruct qwen3-coder-480b-a35b-instruct	Alibaba Cloud	open	Reference only	47.7%
#64	Exaone 4.0 32B exaone-4-0-32b	LG AI Research	open	Reference only	47%
#65	QwQ 32B-Preview QwQ-32B-Preview	Alibaba Cloud	—	Reference only	45.3%
#66	Mistral Medium 3 mistral-medium-3	Mistral AI	closed	Reference only	44%
#67	GPT-4.1 gpt-4-1	OpenAI	closed	Reference only	43.7%
#68	Gemini 2.5 Flash Preview (Non-reasoning) gemini-2-5-flash-04-2025	Google	—	Reference only	43.3%
#69	GPT-4.1 mini gpt-4-1-mini	OpenAI	closed	Reference only	43%
#70	Claude 4 Sonnet claude-4-sonnet	Anthropic	closed	Reference only	40.7%
#71	Solar Pro 2 solar-pro-2	Upstage	closed	Reference only	40.7%
#72	Llama 4 Maverick llama-4-maverick	Meta	open	Reference only	39%
#73	GPT-5 (minimal) gpt-5-minimal	OpenAI	—	Reference only	36.7%
#74	Gemini 2.0 Pro Experimental (Feb '25) gemini-2-0-pro-experimental-02-05	Google	—	Reference only	36%
#75	DeepSeek R1 Distill Llama 8B deepseek-r1-distill-llama-8b	DeepSeek	—	Reference only	33.3%
#76	Gemini 2.0 Flash (Feb '25) gemini-2-0-flash	Google	—	Reference only	33%
#77	Grok 3 grok-3	xAI	—	Reference only	33%
#78	GPT-4o (March 2025, chatgpt-4o-latest) gpt-4o-chatgpt-03-25	OpenAI	—	Reference only	32.7%
#79	Qwen3 235B A22B (Non-reasoning) qwen3-235b-a22b-instruct	Alibaba Cloud	—	Reference only	32.7%
#80	Mistral Small 3.2 mistral-small-3-2	Mistral AI	—	Reference only	32.3%
#81	Gemini 2.0 Flash-Lite (Preview) gemini-2-0-flash-lite-preview	Google	—	Reference only	30.3%
#82	Qwen3 32B (Non-reasoning) qwen3-32b-instruct	Alibaba Cloud	—	Reference only	30.3%
#83	Gemini 2.0 Flash (experimental) gemini-2-0-flash-experimental	Google	—	Reference only	30%
#84	Llama 3.3 Instruct 70B llama-3-3-instruct-70b	Meta	—	Reference only	30%
#85	Qwen3 Coder 30B A3B Instruct qwen3-coder-30b-a3b-instruct	Alibaba Cloud	open	Reference only	29.7%
#86	Solar Pro 2 (Preview) (Non-reasoning) solar-pro-2-preview	Upstage	—	Reference only	29.7%
#87	Sonar Pro sonar-pro	Perplexity	closed	Reference only	29%
#88	Llama 4 Scout llama-4-scout	Meta	open	Reference only	28.3%
#89	Qwen3 14B (Non-reasoning) qwen3-14b-instruct	Alibaba Cloud	—	Reference only	28%
#90	Gemini 2.0 Flash-Lite (Feb '25) gemini-2-0-flash-lite-001	Google	—	Reference only	27.7%
#91	Qwen3 30B A3B (Non-reasoning) qwen3-30b-a3b-instruct	Alibaba Cloud	—	Reference only	26%
#92	DeepSeek V3 deepseek-v3	DeepSeek	open	Reference only	25.3%
#93	Gemma 3 27B gemma-3-27b	Google	open	Reference only	25.3%
#94	Llama 3.1 Nemotron Instruct 70B llama-3-1-nemotron-instruct-70b	NVIDIA	—	Reference only	24.7%
#95	Qwen3 8B (Non-reasoning) qwen3-8b-instruct	Alibaba Cloud	—	Reference only	24.3%
#96	GPT-4.1 nano gpt-4-1-nano	OpenAI	closed	Reference only	23.7%
#97	Qwen2.5 Max qwen-2-5-max	Alibaba Cloud	—	Reference only	23.3%
#98	Gemini 1.5 Pro gemini-1-5-pro	Google	closed	Reference only	23%
#99	Claude 3.7 Sonnet (Non-reasoning) claude-3-7-sonnet	Anthropic	—	Reference only	22.3%
#100	Gemma 3 12B Instruct gemma-3-12b	Google	—	Reference only	22%
#101	Llama 3.1 Instruct 405B llama-3-1-instruct-405b	Meta	—	Reference only	21.3%
#102	Qwen3 4B (Non-reasoning) qwen3-4b-instruct	Alibaba Cloud	—	Reference only	21.3%
#103	Sarvam M (Reasoning) sarvam-m-reasoning	Sarvam	—	Reference only	20.3%
#104	Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) llama-3-3-nemotron-super-49b	NVIDIA	—	Reference only	19.3%
#105	Gemini 1.5 Flash (Sep '24) gemini-1-5-flash	Google	—	Reference only	18%
#106	DeepSeek R1 Distill Qwen 1.5B deepseek-r1-distill-qwen-1-5b	DeepSeek	—	Reference only	17.7%
#107	Llama 3.1 Instruct 70B llama-3-1-instruct-70b	Meta	—	Reference only	17.3%
#108	Nova Premier nova-premier	Amazon	—	Reference only	17%
#109	Qwen2.5 Instruct 72B qwen2-5-72b-instruct	Alibaba Cloud	open	Reference only	16%
#110	Claude 3.5 Sonnet (Oct '24) claude-35-sonnet	Anthropic	—	Reference only	15.7%
#111	GPT-4 Turbo gpt-4-turbo	OpenAI	closed	Reference only	15%
#112	GPT-4o gpt-4o	OpenAI	closed	Reference only	15%
#113	Qwen2 Instruct 72B qwen2-72b-instruct	Alibaba Cloud	—	Reference only	14.7%
#114	Phi-4 phi-4	Microsoft	open	Reference only	14.3%
#115	Gemma 3n E4B Instruct gemma-3n-e4b	Google	—	Reference only	13.7%
#116	Llama Nemotron Super 49B v1.5 (Non-reasoning) llama-nemotron-super-49b-v1-5	NVIDIA	—	Reference only	13.7%
#117	Grok 2 (Dec '24) grok-2-1212	xAI	—	Reference only	13.3%
#118	Llama 3.1 Tulu3 405B tulu3-405b	Allen Institute for AI	—	Reference only	13.3%
#119	Mistral Saba mistral-saba	Mistral AI	—	Reference only	13%
#120	Qwen2.5 Coder 32B Instruct qwen2-5-coder-32b-instruct	Alibaba Cloud	open	Reference only	12%
#121	Qwen2.5 Turbo qwen-turbo	Alibaba Cloud	closed	Reference only	12%
#122	GPT-4o (Aug '24) gpt-4o-2024-08-06	OpenAI	closed	Reference only	11.7%
#123	GPT-4o mini gpt-4o-mini	OpenAI	closed	Reference only	11.7%
#124	GPT-4o (May '24) gpt-4o-2024-05-13	OpenAI	closed	Reference only	11%
#125	Mistral Large 2 mistral-large-2	Mistral AI	closed	Reference only	11%
#126	Qwen2.5 Instruct 32B qwen2.5-32b-instruct	Alibaba Cloud	open	Reference only	11%
#127	Gemma 3n E4B Instruct Preview (May '25) gemma-3n-e4b-preview-0520	Google	—	Reference only	10.7%
#128	Nova Lite nova-lite	Amazon	—	Reference only	10.7%
#129	Nova Pro nova-pro	Amazon	closed	Reference only	10.7%
#130	GPT-4o (ChatGPT) gpt-4o-chatgpt	OpenAI	—	Reference only	10.3%
#131	Grok Beta grok-beta	xAI	—	Reference only	10.3%
#132	Qwen3 0.6B (Reasoning) qwen3-0.6b-instruct-reasoning	Alibaba Cloud	—	Reference only	10%
#133	Claude 3.5 Sonnet (June '24) claude-35-sonnet-june-24	Anthropic	—	Reference only	9.7%
#134	Command A+ command-a-plus	Cohere	open	Reference only	9.7%
#135	Qwen3 1.7B (Non-reasoning) qwen3-1.7b-instruct	Alibaba Cloud	—	Reference only	9.7%
#136	Gemini 1.5 Flash (May '24) gemini-1-5-flash-may-2024	Google	—	Reference only	9.3%
#137	Llama 3.2 Instruct 11B (Vision) llama-3-2-instruct-11b-vision	Meta	—	Reference only	9.3%
#138	Mistral Large 2 (Jul '24) mistral-large-2407	Mistral AI	—	Reference only	9.3%
#139	Mistral Small 3.1 mistral-small-3-1	Mistral AI	—	Reference only	9.3%
#140	Phi-4 Multimodal Instruct phi-4-multimodal	Microsoft	—	Reference only	9.3%
#141	Gemma 3n E2B Instruct gemma-3n-e2b	Google	—	Reference only	9%
#142	Gemini 1.5 Pro (May '24) gemini-1-5-pro-may-2024	Google	—	Reference only	8%
#143	Mistral Small 3 mistral-small-3	Mistral AI	—	Reference only	8%
#144	Nova Micro nova-micro	Amazon	—	Reference only	8%
#145	Llama 3.1 Instruct 8B llama-3-1-instruct-8b	Meta	—	Reference only	7.7%
#146	Pixtral Large pixtral-large-2411	Mistral AI	—	Reference only	7%
#147	Devstral Medium devstral-medium	Mistral AI	—	Reference only	6.7%
#148	Devstral Small (May '25) devstral-small-2505	Mistral AI	open	Reference only	6.7%
#149	Llama 3.2 Instruct 3B llama-3-2-instruct-3b	Meta	—	Reference only	6.7%
#150	Gemma 3 4B Instruct gemma-3-4b	Google	—	Reference only	6.3%
#151	Mistral Small (Sep '24) mistral-small	Mistral AI	—	Reference only	6.3%
#152	Jamba 1.7 Large jamba-1-7-large	AI21 Labs	—	Reference only	5.7%
#153	Qwen2.5 Coder Instruct 7B qwen2-5-coder-7b-instruct	Alibaba Cloud	—	Reference only	5.3%
#154	Llama 3.2 Instruct 90B (Vision) llama-3-2-instruct-90b-vision	Meta	—	Reference only	5%
#155	Claude 3 Sonnet claude-3-sonnet	Anthropic	—	Reference only	4.7%
#156	DeepHermes 3 - Mistral 24B Preview (Non-reasoning) deephermes-3-mistral-24b-preview	Nous Research	—	Reference only	4.7%
#157	Granite 3.3 8B (Non-reasoning) granite-3-3-8b-instruct	IBM	—	Reference only	4.7%
#158	Jamba 1.5 Large jamba-1-5-large	AI21 Labs	—	Reference only	4.7%
#159	Jamba 1.6 Large jamba-1-6-large	AI21 Labs	—	Reference only	4.7%
#160	Phi-3 Mini Instruct 3.8B phi-3-mini	Microsoft	—	Reference only	4%
#161	Mistral Medium mistral-medium	Mistral AI	—	Reference only	3.7%
#162	Claude 2.1 claude-21	Anthropic	—	Reference only	3.3%
#163	Claude 3 Opus claude-3-opus	Anthropic	closed	Reference only	3.3%
#164	Claude 3.5 Haiku claude-3-5-haiku	Anthropic	—	Reference only	3.3%
#165	Gemini 1.5 Flash-8B gemini-1-5-flash-8b	Google	—	Reference only	3.3%
#166	Jamba 1.6 Mini jamba-1-6-mini	AI21 Labs	—	Reference only	3.3%
#167	DBRX Instruct dbrx-instruct	Databricks	open	Reference only	3%
#168	Phi-4 Mini Instruct phi-4-mini	Microsoft	—	Reference only	3%
#169	Hermes 3 - Llama-3.1 70B hermes-3-llama-3-1-70b	Nous Research	—	Reference only	2.3%
#170	LFM 40B lfm-40b	LiquidAI	—	Reference only	2.3%
#171	Llama 2 Chat 13B llama-2-chat-13b	Meta	—	Reference only	1.7%
#172	Qwen3 0.6B (Non-reasoning) qwen3-0.6b-instruct	Alibaba Cloud	—	Reference only	1.7%
#173	Jamba 1.7 Mini jamba-1-7-mini	AI21 Labs	—	Reference only	1.3%
#174	Claude 3 Haiku claude-3-haiku	Anthropic	closed	Reference only	1%
#175	Jamba 1.5 Mini jamba-1-5-mini	AI21 Labs	—	Reference only	1%
#176	Command-R (Mar '24) command-r-03-2024	Cohere	—	Reference only	0.7%
#177	Command-R+ (Apr '24) command-r-plus-04-2024	Cohere	—	Reference only	0.7%
#178	Gemini 1.0 Pro gemini-1-0-pro	Google	closed	Reference only	0.7%
#179	Mistral Small (Feb '24) mistral-small-2402	Mistral AI	—	Reference only	0.7%
#180	Devstral Small (Jul '25) devstral-small	Mistral AI	—	Reference only	0.3%
#181	Claude 2.0 claude-2	Anthropic	—	Reference only	0%
#182	Claude Instant claude-instant	Anthropic	—	Reference only	0%
#183	DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning) deephermes-3-llama-3-1-8b-preview	Nous Research	—	Reference only	0%
#184	Gemma 3 1B Instruct gemma-3-1b	Google	—	Reference only	0%
#185	Llama 2 Chat 70B llama-2-chat-70b	Meta	—	Reference only	0%
#186	Llama 2 Chat 7B llama-2-chat-7b	Meta	—	Reference only	0%
#187	Llama 3 Instruct 70B llama-3-instruct-70b	Meta	—	Reference only	0%
#188	Llama 3 Instruct 8B llama-3-instruct-8b	Meta	—	Reference only	0%
#189	Llama 3.2 Instruct 1B llama-3-2-instruct-1b	Meta	—	Reference only	0%
#190	Mistral 7B Instruct mistral-7b-instruct	Mistral AI	—	Reference only	0%
#191	Mistral Large (Feb '24) mistral-large	Mistral AI	—	Reference only	0%
#192	Mixtral 8x22B Instruct mistral-8x22b-instruct	Mistral AI	—	Reference only	0%
#193	Mixtral 8x7B Instruct mixtral-8x7b-instruct	Mistral AI	—	Reference only	0%
#194	OpenChat 3.5 (1210) openchat-35	OpenChat	—	Reference only	0%

Rank

Model

Provider

License

Evidence use

Score

GPT-5

gpt-5

OpenAI

closed

Reference only

95.7%

Grok 4

grok-4

xAI

closed

Reference only

94.3%

o4-mini

OpenAI

closed

Reference only

94%

Qwen3 235B A22B 2507 (Reasoning)

qwen3-235b-a22b-instruct-2507-reasoning

Alibaba Cloud

—

Reference only

94%

Grok 3 mini Reasoning (high)

grok-3-mini-reasoning

xAI

—

Reference only

93.3%

GPT-5 (medium)

gpt-5-medium

OpenAI

closed

Reference only

91.7%

Qwen3 30B A3B 2507 (Reasoning)

qwen3-30b-a3b-2507-reasoning

Alibaba Cloud

—

Reference only

90.7%

OpenAI

closed

Reference only

90.3%

DeepSeek-R1

deepseek-r1

DeepSeek

open

Reference only

89.3%

#10

Gemini 2.5 Pro

gemini-2.5-pro

Google

closed

Reference only

88.7%

#11

GLM-4.5 (Reasoning)

glm-4.5

Z.AI

open

Reference only

87.3%

#12

Gemini 2.5 Pro Preview (Mar' 25)

gemini-2-5-pro-03-25

Google

—

Reference only

87%

#13

Llama Nemotron Super 49B v1.5 (Reasoning)

llama-nemotron-super-49b-v1-5-reasoning

NVIDIA

—

Reference only

86%

#14

o3-mini (high)

o3-mini-high

OpenAI

—

Reference only

86%

#15

MiniMax M1 80k

minimax-m1-80k

MiniMax

closed

Reference only

84.7%

#16

EXAONE 4.0 32B (Reasoning)

exaone-4-0-32b-reasoning

LG AI Research

—

Reference only

84.3%

#17

Gemini 2.5 Flash Preview (Reasoning)

gemini-2-5-flash-reasoning-04-2025

Google

—

Reference only

84.3%

#18

Gemini 2.5 Pro Preview (May' 25)

gemini-2-5-pro-05-06

Google

—

Reference only

84.3%

#19

Qwen3 235B A22B (Reasoning)

qwen3-235b-a22b-instruct-reasoning

Alibaba Cloud

—

Reference only

84%

#20

GPT-5 (low)

gpt-5-low

OpenAI

—

Reference only

83%

#21

Gemini 2.5 Flash (Reasoning)

gemini-2-5-flash-reasoning

Google

—

Reference only

82.3%

#22

MiniMax M1 40k

minimax-m1-40k

MiniMax

—

Reference only

81.3%

#23

Qwen3 32B (Reasoning)

qwen3-32b-instruct-reasoning

Alibaba Cloud

—

Reference only

80.7%

#24

Sonar Reasoning Pro

sonar-reasoning-pro

Perplexity

closed

Reference only

79%

#25

QwQ 32B

qwq-32b

Alibaba Cloud

—

Reference only

78%

#26

Claude 4 Sonnet (Reasoning)

claude-4-sonnet-thinking

Anthropic

—

Reference only

77.3%

#27

o3-mini

OpenAI

closed

Reference only

77%

#28

Sonar Reasoning

sonar-reasoning

Perplexity

—

Reference only

77%

#29

Qwen3 14B (Reasoning)

qwen3-14b-instruct-reasoning

Alibaba Cloud

—

Reference only

76.3%

#30

Claude 4 Opus (Reasoning)

claude-4-opus-thinking

Anthropic

—

Reference only

75.7%

#31

Qwen3 30B A3B (Reasoning)

qwen3-30b-a3b-instruct-reasoning

Alibaba Cloud

—

Reference only

75.3%

#32

Llama 3.1 Nemotron Ultra 253B v1 (Reasoning)

llama-3-1-nemotron-ultra-253b-v1-reasoning

NVIDIA

—

Reference only

74.7%

#33

Qwen3 8B (Reasoning)

qwen3-8b-instruct-reasoning

Alibaba Cloud

—

Reference only

74.7%

#34

Qwen3 30B A3B 2507 Instruct

qwen3-30b-a3b-2507

Alibaba Cloud

—

Reference only

72.7%

#35

OpenAI

closed

Reference only

72.3%

#36

Qwen3 235B A22B 2507 Instruct

qwen3-235b-a22b-instruct-2507

Alibaba Cloud

—

Reference only

71.7%

#37

Magistral Small 1

magistral-small

Mistral AI

open

Reference only

71.3%

#38

Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning)

llama-3-1-nemotron-nano-4b-reasoning

NVIDIA

—

Reference only

70.7%

#39

Gemini 2.5 Flash-Lite (Reasoning)

gemini-2-5-flash-lite-reasoning

Google

—

Reference only

70.3%

#40

Magistral Medium 1

magistral-medium

Mistral AI

—

Reference only

70%

#41

Kimi K2

kimi-k2

Kimi

—

Reference only

69.3%

#42

Solar Pro 2 (Reasoning)

solar-pro-2-reasoning

Upstage

—

Reference only

69%

#43

DeepSeek R1 Distill Qwen 32B

deepseek-r1-distill-qwen-32b

DeepSeek

open

Reference only

68.7%

#44

DeepSeek R1 (Jan '25)

deepseek-r1-0120

DeepSeek

—

Reference only

68.3%

#45

GLM-4.5-Air

glm-4-5-air

Z.AI

closed

Reference only

67.3%

#46

DeepSeek R1 Distill Llama 70B

deepseek-r1-distill-llama-70b

DeepSeek

—

Reference only

67%

#47

DeepSeek R1 Distill Qwen 14B

deepseek-r1-distill-qwen-14b

DeepSeek

—

Reference only

66.7%

#48

Solar Pro 2 (Preview) (Reasoning)

solar-pro-2-preview-reasoning

Upstage

—

Reference only

66.3%

#49

Qwen3 4B (Reasoning)

qwen3-4b-instruct-reasoning

Alibaba Cloud

—

Reference only

65.7%

#50

DeepSeek R1 0528 Qwen3 8B

deepseek-r1-qwen3-8b

DeepSeek

—

Reference only

65%

#51

o1-mini

OpenAI

—

Reference only

60.3%

#52

Llama 3.3 Nemotron Super 49B v1 (Reasoning)

llama-3-3-nemotron-super-49b-reasoning

NVIDIA

—

Reference only

58.3%

#53

Claude 4 Opus (Non-reasoning)

claude-4-opus

Anthropic

—

Reference only

56.3%

#54

DeepSeek V3 0324

deepseek-v3-0324

DeepSeek

—

Reference only

52%

#55

Qwen3 1.7B (Reasoning)

qwen3-1.7b-instruct-reasoning

Alibaba Cloud

—

Reference only

51%

#56

Reka Flash 3

reka-flash-3

Reka AI

—

Reference only

51%

#57

Gemini 2.0 Flash Thinking Experimental (Jan '25)

gemini-2-0-flash-thinking-exp-0121

Google

—

Reference only

50%

#58

Gemini 2.5 Flash

gemini-2.5-flash

Google

closed

Reference only

50%

#59

Gemini 2.5 Flash-Lite (Non-reasoning)

gemini-2-5-flash-lite

Google

—

Reference only

50%

#60

ERNIE 4.5 300B A47B

ernie-4-5-300b-a47b

Baidu

—

Reference only

49.3%

#61

Claude 3.7 Sonnet (Reasoning)

claude-3-7-sonnet-thinking

Anthropic

—

Reference only

48.7%

#62

Sonar

sonar

Perplexity

closed

Reference only

48.7%

#63

Qwen3 Coder 480B A35B Instruct

qwen3-coder-480b-a35b-instruct

Alibaba Cloud

open

Reference only

47.7%

#64

Exaone 4.0 32B

exaone-4-0-32b

LG AI Research

open

Reference only

47%

#65

QwQ 32B-Preview

QwQ-32B-Preview

Alibaba Cloud

—

Reference only

45.3%

#66

Mistral Medium 3

mistral-medium-3

Mistral AI

closed

Reference only

44%

#67

GPT-4.1

gpt-4-1

OpenAI

closed

Reference only

43.7%

#68

Gemini 2.5 Flash Preview (Non-reasoning)

gemini-2-5-flash-04-2025

Google

—

Reference only

43.3%

#69

GPT-4.1 mini

gpt-4-1-mini

OpenAI

closed

Reference only

43%

#70

Claude 4 Sonnet

claude-4-sonnet

Anthropic

closed

Reference only

40.7%

#71

Solar Pro 2

solar-pro-2

Upstage

closed

Reference only

40.7%

#72

Llama 4 Maverick

llama-4-maverick

Meta

open

Reference only

39%

#73

GPT-5 (minimal)

gpt-5-minimal

OpenAI

—

Reference only

36.7%

#74

Gemini 2.0 Pro Experimental (Feb '25)

gemini-2-0-pro-experimental-02-05

Google

—

Reference only

36%

#75

DeepSeek R1 Distill Llama 8B

deepseek-r1-distill-llama-8b

DeepSeek

—

Reference only

33.3%

#76

Gemini 2.0 Flash (Feb '25)

gemini-2-0-flash

Google

—

Reference only

33%

#77

Grok 3

grok-3

xAI

—

Reference only

33%

#78

GPT-4o (March 2025, chatgpt-4o-latest)

gpt-4o-chatgpt-03-25

OpenAI

—

Reference only

32.7%

#79

Qwen3 235B A22B (Non-reasoning)

qwen3-235b-a22b-instruct

Alibaba Cloud

—

Reference only

32.7%

#80

Mistral Small 3.2

mistral-small-3-2

Mistral AI

—

Reference only

32.3%

#81

Gemini 2.0 Flash-Lite (Preview)

gemini-2-0-flash-lite-preview

Google

—

Reference only

30.3%

#82

Qwen3 32B (Non-reasoning)

qwen3-32b-instruct

Alibaba Cloud

—

Reference only

30.3%

#83

Gemini 2.0 Flash (experimental)

gemini-2-0-flash-experimental

Google

—

Reference only

30%

#84

Llama 3.3 Instruct 70B

llama-3-3-instruct-70b

Meta

—

Reference only

30%

#85

Qwen3 Coder 30B A3B Instruct

qwen3-coder-30b-a3b-instruct

Alibaba Cloud

open

Reference only

29.7%

#86

Solar Pro 2 (Preview) (Non-reasoning)

solar-pro-2-preview

Upstage

—

Reference only

29.7%

#87

Sonar Pro

sonar-pro

Perplexity

closed

Reference only

29%

#88

Llama 4 Scout

llama-4-scout

Meta

open

Reference only

28.3%

#89

Qwen3 14B (Non-reasoning)

qwen3-14b-instruct

Alibaba Cloud

—

Reference only

28%

#90

Gemini 2.0 Flash-Lite (Feb '25)

gemini-2-0-flash-lite-001

Google

—

Reference only

27.7%

#91

Qwen3 30B A3B (Non-reasoning)

qwen3-30b-a3b-instruct

Alibaba Cloud

—

Reference only

26%

#92

DeepSeek V3

deepseek-v3

DeepSeek

open

Reference only

25.3%

#93

Gemma 3 27B

gemma-3-27b

Google

open

Reference only

25.3%

#94

Llama 3.1 Nemotron Instruct 70B

llama-3-1-nemotron-instruct-70b

NVIDIA

—

Reference only

24.7%

#95

Qwen3 8B (Non-reasoning)

qwen3-8b-instruct

Alibaba Cloud

—

Reference only

24.3%

#96

GPT-4.1 nano

gpt-4-1-nano

OpenAI

closed

Reference only

23.7%

#97

Qwen2.5 Max

qwen-2-5-max

Alibaba Cloud

—

Reference only

23.3%

#98

Gemini 1.5 Pro

gemini-1-5-pro

Google

closed

Reference only

23%

#99

Claude 3.7 Sonnet (Non-reasoning)

claude-3-7-sonnet

Anthropic

—

Reference only

22.3%

#100

Gemma 3 12B Instruct

gemma-3-12b

Google

—

Reference only

22%

#101

Llama 3.1 Instruct 405B

llama-3-1-instruct-405b

Meta

—

Reference only

21.3%

#102

Qwen3 4B (Non-reasoning)

qwen3-4b-instruct

Alibaba Cloud

—

Reference only

21.3%

#103

Sarvam M (Reasoning)

sarvam-m-reasoning

Sarvam

—

Reference only

20.3%

#104

Llama 3.3 Nemotron Super 49B v1 (Non-reasoning)

llama-3-3-nemotron-super-49b

NVIDIA

—

Reference only

19.3%

#105

Gemini 1.5 Flash (Sep '24)

gemini-1-5-flash

Google

—

Reference only

18%

#106

DeepSeek R1 Distill Qwen 1.5B

deepseek-r1-distill-qwen-1-5b

DeepSeek

—

Reference only

17.7%

#107

Llama 3.1 Instruct 70B

llama-3-1-instruct-70b

Meta

—

Reference only

17.3%

#108

Nova Premier

nova-premier

Amazon

—

Reference only

17%

#109

Qwen2.5 Instruct 72B

qwen2-5-72b-instruct

Alibaba Cloud

open

Reference only

16%

#110

Claude 3.5 Sonnet (Oct '24)

claude-35-sonnet

Anthropic

—

Reference only

15.7%

#111

GPT-4 Turbo

gpt-4-turbo

OpenAI

closed

Reference only

15%

#112

GPT-4o

gpt-4o

OpenAI

closed

Reference only

15%

#113

Qwen2 Instruct 72B

qwen2-72b-instruct

Alibaba Cloud

—

Reference only

14.7%

#114

Phi-4

phi-4

Microsoft

open

Reference only

14.3%

#115

Gemma 3n E4B Instruct

gemma-3n-e4b

Google

—

Reference only

13.7%

#116

Llama Nemotron Super 49B v1.5 (Non-reasoning)

llama-nemotron-super-49b-v1-5

NVIDIA

—

Reference only

13.7%

#117

Grok 2 (Dec '24)

grok-2-1212

xAI

—

Reference only

13.3%

#118

Llama 3.1 Tulu3 405B

tulu3-405b

Allen Institute for AI

—

Reference only

13.3%

#119

Mistral Saba

mistral-saba

Mistral AI

—

Reference only

13%

#120

Qwen2.5 Coder 32B Instruct

qwen2-5-coder-32b-instruct

Alibaba Cloud

open

Reference only

12%

#121

Qwen2.5 Turbo

qwen-turbo

Alibaba Cloud

closed

Reference only

12%

#122

GPT-4o (Aug '24)

gpt-4o-2024-08-06

OpenAI

closed

Reference only

11.7%

#123

GPT-4o mini

gpt-4o-mini

OpenAI

closed

Reference only

11.7%

#124

GPT-4o (May '24)

gpt-4o-2024-05-13

OpenAI

closed

Reference only

11%

#125

Mistral Large 2

mistral-large-2

Mistral AI

closed

Reference only

11%

#126

Qwen2.5 Instruct 32B

qwen2.5-32b-instruct

Alibaba Cloud

open

Reference only

11%

#127

Gemma 3n E4B Instruct Preview (May '25)

gemma-3n-e4b-preview-0520

Google

—

Reference only

10.7%

#128

Nova Lite

nova-lite

Amazon

—

Reference only

10.7%

#129

Nova Pro

nova-pro

Amazon

closed

Reference only

10.7%

#130

GPT-4o (ChatGPT)

gpt-4o-chatgpt

OpenAI

—

Reference only

10.3%

#131

Grok Beta

grok-beta

xAI

—

Reference only

10.3%

#132

Qwen3 0.6B (Reasoning)

qwen3-0.6b-instruct-reasoning

Alibaba Cloud

—

Reference only

10%

#133

Claude 3.5 Sonnet (June '24)

claude-35-sonnet-june-24

Anthropic

—

Reference only

9.7%

#134

Command A+

command-a-plus

Cohere

open

Reference only

9.7%

#135

Qwen3 1.7B (Non-reasoning)

qwen3-1.7b-instruct

Alibaba Cloud

—

Reference only

9.7%

#136

Gemini 1.5 Flash (May '24)

gemini-1-5-flash-may-2024

Google

—

Reference only

9.3%

#137

Llama 3.2 Instruct 11B (Vision)

llama-3-2-instruct-11b-vision

Meta

—

Reference only

9.3%

#138

Mistral Large 2 (Jul '24)

mistral-large-2407

Mistral AI

—

Reference only

9.3%

#139

Mistral Small 3.1

mistral-small-3-1

Mistral AI

—

Reference only

9.3%

#140

Phi-4 Multimodal Instruct

phi-4-multimodal

Microsoft

—

Reference only

9.3%

#141

Gemma 3n E2B Instruct

gemma-3n-e2b

Google

—

Reference only

#142

Gemini 1.5 Pro (May '24)

gemini-1-5-pro-may-2024

Google

—

Reference only

#143

Mistral Small 3

mistral-small-3

Mistral AI

—

Reference only

#144

Nova Micro

nova-micro

Amazon

—

Reference only

#145

Llama 3.1 Instruct 8B

llama-3-1-instruct-8b

Meta

—

Reference only

7.7%

#146

Pixtral Large

pixtral-large-2411

Mistral AI

—

Reference only

#147

Devstral Medium

devstral-medium

Mistral AI

—

Reference only

6.7%

#148

Devstral Small (May '25)

devstral-small-2505

Mistral AI

open

Reference only

6.7%

#149

Llama 3.2 Instruct 3B

llama-3-2-instruct-3b

Meta

—

Reference only

6.7%

#150

Gemma 3 4B Instruct

gemma-3-4b

Google

—

Reference only

6.3%

#151

Mistral Small (Sep '24)

mistral-small

Mistral AI

—

Reference only

6.3%

#152

Jamba 1.7 Large

jamba-1-7-large

AI21 Labs

—

Reference only

5.7%

#153

Qwen2.5 Coder Instruct 7B

qwen2-5-coder-7b-instruct

Alibaba Cloud

—

Reference only

5.3%

#154

Llama 3.2 Instruct 90B (Vision)

llama-3-2-instruct-90b-vision

Meta

—

Reference only

#155

Claude 3 Sonnet

claude-3-sonnet

Anthropic

—

Reference only

4.7%

#156

DeepHermes 3 - Mistral 24B Preview (Non-reasoning)

deephermes-3-mistral-24b-preview

Nous Research

—

Reference only

4.7%

#157

Granite 3.3 8B (Non-reasoning)

granite-3-3-8b-instruct

IBM

—

Reference only

4.7%

#158

Jamba 1.5 Large

jamba-1-5-large

AI21 Labs

—

Reference only

4.7%

#159

Jamba 1.6 Large

jamba-1-6-large

AI21 Labs

—

Reference only

4.7%

#160

Phi-3 Mini Instruct 3.8B

phi-3-mini

Microsoft

—

Reference only

#161

Mistral Medium

mistral-medium

Mistral AI

—

Reference only

3.7%

#162

Claude 2.1

claude-21

Anthropic

—

Reference only

3.3%

#163

Claude 3 Opus

claude-3-opus

Anthropic

closed

Reference only

3.3%

#164

Claude 3.5 Haiku

claude-3-5-haiku

Anthropic

—

Reference only

3.3%

#165

Gemini 1.5 Flash-8B

gemini-1-5-flash-8b

Google

—

Reference only

3.3%

#166

Jamba 1.6 Mini

jamba-1-6-mini

AI21 Labs

—

Reference only

3.3%

#167

DBRX Instruct

dbrx-instruct

Databricks

open

Reference only

#168

Phi-4 Mini Instruct

phi-4-mini

Microsoft

—

Reference only

#169

Hermes 3 - Llama-3.1 70B

hermes-3-llama-3-1-70b

Nous Research

—

Reference only

2.3%

#170

LFM 40B

lfm-40b

LiquidAI

—

Reference only

2.3%

#171

Llama 2 Chat 13B

llama-2-chat-13b

Meta

—

Reference only

1.7%

#172

Qwen3 0.6B (Non-reasoning)

qwen3-0.6b-instruct

Alibaba Cloud

—

Reference only

1.7%

#173

Jamba 1.7 Mini

jamba-1-7-mini

AI21 Labs

—

Reference only

1.3%

#174

Claude 3 Haiku

claude-3-haiku

Anthropic

closed

Reference only

#175

Jamba 1.5 Mini

jamba-1-5-mini

AI21 Labs

—

Reference only

#176

Command-R (Mar '24)

command-r-03-2024

Cohere

—

Reference only

0.7%

#177

Command-R+ (Apr '24)

command-r-plus-04-2024

Cohere

—

Reference only

0.7%

#178

Gemini 1.0 Pro

gemini-1-0-pro

Google

closed

Reference only

0.7%

#179

Mistral Small (Feb '24)

mistral-small-2402

Mistral AI

—

Reference only

0.7%

#180

Devstral Small (Jul '25)

devstral-small

Mistral AI

—

Reference only

0.3%

#181

Claude 2.0

claude-2

Anthropic

—

Reference only

#182

Claude Instant

claude-instant

Anthropic

—

Reference only

#183

DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning)

deephermes-3-llama-3-1-8b-preview

Nous Research

—

Reference only

#184

Gemma 3 1B Instruct

gemma-3-1b

Google

—

Reference only

#185

Llama 2 Chat 70B

llama-2-chat-70b

Meta

—

Reference only

#186

Llama 2 Chat 7B

llama-2-chat-7b

Meta

—

Reference only

#187

Llama 3 Instruct 70B

llama-3-instruct-70b

Meta

—

Reference only

#188

Llama 3 Instruct 8B

llama-3-instruct-8b

Meta

—

Reference only

#189

Llama 3.2 Instruct 1B

llama-3-2-instruct-1b

Meta

—

Reference only

#190

Mistral 7B Instruct

mistral-7b-instruct

Mistral AI

—

Reference only

#191

Mistral Large (Feb '24)

mistral-large

Mistral AI

—

Reference only

#192

Mixtral 8x22B Instruct

mistral-8x22b-instruct

Mistral AI

—

Reference only

#193

Mixtral 8x7B Instruct

mixtral-8x7b-instruct

Mistral AI

—

Reference only

#194

OpenChat 3.5 (1210)

openchat-35

OpenChat

—

Reference only

Definition and scoring

Organisation

Mathematical Association of America

What does American Invitational Mathematics Examination 2024 measure?

The 2024 edition of AIME, maintaining the same format of 15 challenging mathematics problems with integer answers from 000 to 999.

Which model scores highest on American Invitational Mathematics Examination 2024?

GPT-5 by OpenAI currently leads with 95.7%.

How many models are evaluated on American Invitational Mathematics Examination 2024?

194 models in the LuminaBench cohort have a qualifying score on this benchmark.

Does this affect overall Lumina rank?

No. This benchmark is display-only and does not enter the overall Lumina composite.

Accessible data for American Invitational Mathematics Examination 2024 leaderboard
Model	Rank	Provider	Score
GPT-5	#1	OpenAI	95.7%
Grok 4	#2	xAI	94.3%
o4-mini	#3	OpenAI	94%
Qwen3 235B A22B 2507 (Reasoning)	#4	Alibaba Cloud	94%
Grok 3 mini Reasoning (high)	#5	SpaceXAI	93.3%
GPT-5 (medium)	#6	OpenAI	91.7%
Qwen3 30B A3B 2507 (Reasoning)	#7	Alibaba Cloud	90.7%
o3	#8	OpenAI	90.3%
DeepSeek-R1	#9	DeepSeek	89.3%
Gemini 2.5 Pro	#10	Google	88.7%

Model

Rank

Provider

Score

GPT-5

OpenAI

95.7%

Grok 4

xAI

94.3%

o4-mini

OpenAI

94%

Qwen3 235B A22B 2507 (Reasoning)

Alibaba Cloud

94%

Grok 3 mini Reasoning (high)

SpaceXAI

93.3%

GPT-5 (medium)

OpenAI

91.7%

Qwen3 30B A3B 2507 (Reasoning)

Alibaba Cloud

90.7%

OpenAI

90.3%

DeepSeek-R1

DeepSeek

89.3%

Gemini 2.5 Pro

#10

Google

88.7%

American Invitational Mathematics Examination 2024 leaderboard ranked by score
Rank	Model	Provider	License	Evidence use	Score
#1	GPT-5 gpt-5	OpenAI	closed	Reference only	95.7%
#2	Grok 4 grok-4	xAI	closed	Reference only	94.3%
#3	o4-mini o4-mini	OpenAI	closed	Reference only	94%
#4	Qwen3 235B A22B 2507 (Reasoning) qwen3-235b-a22b-instruct-2507-reasoning	Alibaba Cloud	—	Reference only	94%
#5	Grok 3 mini Reasoning (high) grok-3-mini-reasoning	xAI	—	Reference only	93.3%
#6	GPT-5 (medium) gpt-5-medium	OpenAI	closed	Reference only	91.7%
#7	Qwen3 30B A3B 2507 (Reasoning) qwen3-30b-a3b-2507-reasoning	Alibaba Cloud	—	Reference only	90.7%
#8	o3 o3	OpenAI	closed	Reference only	90.3%
#9	DeepSeek-R1 deepseek-r1	DeepSeek	open	Reference only	89.3%
#10	Gemini 2.5 Pro gemini-2.5-pro	Google	closed	Reference only	88.7%
#11	GLM-4.5 (Reasoning) glm-4.5	Z.AI	open	Reference only	87.3%
#12	Gemini 2.5 Pro Preview (Mar' 25) gemini-2-5-pro-03-25	Google	—	Reference only	87%
#13	Llama Nemotron Super 49B v1.5 (Reasoning) llama-nemotron-super-49b-v1-5-reasoning	NVIDIA	—	Reference only	86%
#14	o3-mini (high) o3-mini-high	OpenAI	—	Reference only	86%
#15	MiniMax M1 80k minimax-m1-80k	MiniMax	closed	Reference only	84.7%
#16	EXAONE 4.0 32B (Reasoning) exaone-4-0-32b-reasoning	LG AI Research	—	Reference only	84.3%
#17	Gemini 2.5 Flash Preview (Reasoning) gemini-2-5-flash-reasoning-04-2025	Google	—	Reference only	84.3%
#18	Gemini 2.5 Pro Preview (May' 25) gemini-2-5-pro-05-06	Google	—	Reference only	84.3%
#19	Qwen3 235B A22B (Reasoning) qwen3-235b-a22b-instruct-reasoning	Alibaba Cloud	—	Reference only	84%
#20	GPT-5 (low) gpt-5-low	OpenAI	—	Reference only	83%
#21	Gemini 2.5 Flash (Reasoning) gemini-2-5-flash-reasoning	Google	—	Reference only	82.3%
#22	MiniMax M1 40k minimax-m1-40k	MiniMax	—	Reference only	81.3%
#23	Qwen3 32B (Reasoning) qwen3-32b-instruct-reasoning	Alibaba Cloud	—	Reference only	80.7%
#24	Sonar Reasoning Pro sonar-reasoning-pro	Perplexity	closed	Reference only	79%
#25	QwQ 32B qwq-32b	Alibaba Cloud	—	Reference only	78%
#26	Claude 4 Sonnet (Reasoning) claude-4-sonnet-thinking	Anthropic	—	Reference only	77.3%
#27	o3-mini o3-mini	OpenAI	closed	Reference only	77%
#28	Sonar Reasoning sonar-reasoning	Perplexity	—	Reference only	77%
#29	Qwen3 14B (Reasoning) qwen3-14b-instruct-reasoning	Alibaba Cloud	—	Reference only	76.3%
#30	Claude 4 Opus (Reasoning) claude-4-opus-thinking	Anthropic	—	Reference only	75.7%
#31	Qwen3 30B A3B (Reasoning) qwen3-30b-a3b-instruct-reasoning	Alibaba Cloud	—	Reference only	75.3%
#32	Llama 3.1 Nemotron Ultra 253B v1 (Reasoning) llama-3-1-nemotron-ultra-253b-v1-reasoning	NVIDIA	—	Reference only	74.7%
#33	Qwen3 8B (Reasoning) qwen3-8b-instruct-reasoning	Alibaba Cloud	—	Reference only	74.7%
#34	Qwen3 30B A3B 2507 Instruct qwen3-30b-a3b-2507	Alibaba Cloud	—	Reference only	72.7%
#35	o1 o1	OpenAI	closed	Reference only	72.3%
#36	Qwen3 235B A22B 2507 Instruct qwen3-235b-a22b-instruct-2507	Alibaba Cloud	—	Reference only	71.7%
#37	Magistral Small 1 magistral-small	Mistral AI	open	Reference only	71.3%
#38	Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning) llama-3-1-nemotron-nano-4b-reasoning	NVIDIA	—	Reference only	70.7%
#39	Gemini 2.5 Flash-Lite (Reasoning) gemini-2-5-flash-lite-reasoning	Google	—	Reference only	70.3%
#40	Magistral Medium 1 magistral-medium	Mistral AI	—	Reference only	70%
#41	Kimi K2 kimi-k2	Kimi	—	Reference only	69.3%
#42	Solar Pro 2 (Reasoning) solar-pro-2-reasoning	Upstage	—	Reference only	69%
#43	DeepSeek R1 Distill Qwen 32B deepseek-r1-distill-qwen-32b	DeepSeek	open	Reference only	68.7%
#44	DeepSeek R1 (Jan '25) deepseek-r1-0120	DeepSeek	—	Reference only	68.3%
#45	GLM-4.5-Air glm-4-5-air	Z.AI	closed	Reference only	67.3%
#46	DeepSeek R1 Distill Llama 70B deepseek-r1-distill-llama-70b	DeepSeek	—	Reference only	67%
#47	DeepSeek R1 Distill Qwen 14B deepseek-r1-distill-qwen-14b	DeepSeek	—	Reference only	66.7%
#48	Solar Pro 2 (Preview) (Reasoning) solar-pro-2-preview-reasoning	Upstage	—	Reference only	66.3%
#49	Qwen3 4B (Reasoning) qwen3-4b-instruct-reasoning	Alibaba Cloud	—	Reference only	65.7%
#50	DeepSeek R1 0528 Qwen3 8B deepseek-r1-qwen3-8b	DeepSeek	—	Reference only	65%
#51	o1-mini o1-mini	OpenAI	—	Reference only	60.3%
#52	Llama 3.3 Nemotron Super 49B v1 (Reasoning) llama-3-3-nemotron-super-49b-reasoning	NVIDIA	—	Reference only	58.3%
#53	Claude 4 Opus (Non-reasoning) claude-4-opus	Anthropic	—	Reference only	56.3%
#54	DeepSeek V3 0324 deepseek-v3-0324	DeepSeek	—	Reference only	52%
#55	Qwen3 1.7B (Reasoning) qwen3-1.7b-instruct-reasoning	Alibaba Cloud	—	Reference only	51%
#56	Reka Flash 3 reka-flash-3	Reka AI	—	Reference only	51%
#57	Gemini 2.0 Flash Thinking Experimental (Jan '25) gemini-2-0-flash-thinking-exp-0121	Google	—	Reference only	50%
#58	Gemini 2.5 Flash gemini-2.5-flash	Google	closed	Reference only	50%
#59	Gemini 2.5 Flash-Lite (Non-reasoning) gemini-2-5-flash-lite	Google	—	Reference only	50%
#60	ERNIE 4.5 300B A47B ernie-4-5-300b-a47b	Baidu	—	Reference only	49.3%
#61	Claude 3.7 Sonnet (Reasoning) claude-3-7-sonnet-thinking	Anthropic	—	Reference only	48.7%
#62	Sonar sonar	Perplexity	closed	Reference only	48.7%
#63	Qwen3 Coder 480B A35B Instruct qwen3-coder-480b-a35b-instruct	Alibaba Cloud	open	Reference only	47.7%
#64	Exaone 4.0 32B exaone-4-0-32b	LG AI Research	open	Reference only	47%
#65	QwQ 32B-Preview QwQ-32B-Preview	Alibaba Cloud	—	Reference only	45.3%
#66	Mistral Medium 3 mistral-medium-3	Mistral AI	closed	Reference only	44%
#67	GPT-4.1 gpt-4-1	OpenAI	closed	Reference only	43.7%
#68	Gemini 2.5 Flash Preview (Non-reasoning) gemini-2-5-flash-04-2025	Google	—	Reference only	43.3%
#69	GPT-4.1 mini gpt-4-1-mini	OpenAI	closed	Reference only	43%
#70	Claude 4 Sonnet claude-4-sonnet	Anthropic	closed	Reference only	40.7%
#71	Solar Pro 2 solar-pro-2	Upstage	closed	Reference only	40.7%
#72	Llama 4 Maverick llama-4-maverick	Meta	open	Reference only	39%
#73	GPT-5 (minimal) gpt-5-minimal	OpenAI	—	Reference only	36.7%
#74	Gemini 2.0 Pro Experimental (Feb '25) gemini-2-0-pro-experimental-02-05	Google	—	Reference only	36%
#75	DeepSeek R1 Distill Llama 8B deepseek-r1-distill-llama-8b	DeepSeek	—	Reference only	33.3%
#76	Gemini 2.0 Flash (Feb '25) gemini-2-0-flash	Google	—	Reference only	33%
#77	Grok 3 grok-3	xAI	—	Reference only	33%
#78	GPT-4o (March 2025, chatgpt-4o-latest) gpt-4o-chatgpt-03-25	OpenAI	—	Reference only	32.7%
#79	Qwen3 235B A22B (Non-reasoning) qwen3-235b-a22b-instruct	Alibaba Cloud	—	Reference only	32.7%
#80	Mistral Small 3.2 mistral-small-3-2	Mistral AI	—	Reference only	32.3%
#81	Gemini 2.0 Flash-Lite (Preview) gemini-2-0-flash-lite-preview	Google	—	Reference only	30.3%
#82	Qwen3 32B (Non-reasoning) qwen3-32b-instruct	Alibaba Cloud	—	Reference only	30.3%
#83	Gemini 2.0 Flash (experimental) gemini-2-0-flash-experimental	Google	—	Reference only	30%
#84	Llama 3.3 Instruct 70B llama-3-3-instruct-70b	Meta	—	Reference only	30%
#85	Qwen3 Coder 30B A3B Instruct qwen3-coder-30b-a3b-instruct	Alibaba Cloud	open	Reference only	29.7%
#86	Solar Pro 2 (Preview) (Non-reasoning) solar-pro-2-preview	Upstage	—	Reference only	29.7%
#87	Sonar Pro sonar-pro	Perplexity	closed	Reference only	29%
#88	Llama 4 Scout llama-4-scout	Meta	open	Reference only	28.3%
#89	Qwen3 14B (Non-reasoning) qwen3-14b-instruct	Alibaba Cloud	—	Reference only	28%
#90	Gemini 2.0 Flash-Lite (Feb '25) gemini-2-0-flash-lite-001	Google	—	Reference only	27.7%
#91	Qwen3 30B A3B (Non-reasoning) qwen3-30b-a3b-instruct	Alibaba Cloud	—	Reference only	26%
#92	DeepSeek V3 deepseek-v3	DeepSeek	open	Reference only	25.3%
#93	Gemma 3 27B gemma-3-27b	Google	open	Reference only	25.3%
#94	Llama 3.1 Nemotron Instruct 70B llama-3-1-nemotron-instruct-70b	NVIDIA	—	Reference only	24.7%
#95	Qwen3 8B (Non-reasoning) qwen3-8b-instruct	Alibaba Cloud	—	Reference only	24.3%
#96	GPT-4.1 nano gpt-4-1-nano	OpenAI	closed	Reference only	23.7%
#97	Qwen2.5 Max qwen-2-5-max	Alibaba Cloud	—	Reference only	23.3%
#98	Gemini 1.5 Pro gemini-1-5-pro	Google	closed	Reference only	23%
#99	Claude 3.7 Sonnet (Non-reasoning) claude-3-7-sonnet	Anthropic	—	Reference only	22.3%
#100	Gemma 3 12B Instruct gemma-3-12b	Google	—	Reference only	22%
#101	Llama 3.1 Instruct 405B llama-3-1-instruct-405b	Meta	—	Reference only	21.3%
#102	Qwen3 4B (Non-reasoning) qwen3-4b-instruct	Alibaba Cloud	—	Reference only	21.3%
#103	Sarvam M (Reasoning) sarvam-m-reasoning	Sarvam	—	Reference only	20.3%
#104	Llama 3.3 Nemotron Super 49B v1 (Non-reasoning) llama-3-3-nemotron-super-49b	NVIDIA	—	Reference only	19.3%
#105	Gemini 1.5 Flash (Sep '24) gemini-1-5-flash	Google	—	Reference only	18%
#106	DeepSeek R1 Distill Qwen 1.5B deepseek-r1-distill-qwen-1-5b	DeepSeek	—	Reference only	17.7%
#107	Llama 3.1 Instruct 70B llama-3-1-instruct-70b	Meta	—	Reference only	17.3%
#108	Nova Premier nova-premier	Amazon	—	Reference only	17%
#109	Qwen2.5 Instruct 72B qwen2-5-72b-instruct	Alibaba Cloud	open	Reference only	16%
#110	Claude 3.5 Sonnet (Oct '24) claude-35-sonnet	Anthropic	—	Reference only	15.7%
#111	GPT-4 Turbo gpt-4-turbo	OpenAI	closed	Reference only	15%
#112	GPT-4o gpt-4o	OpenAI	closed	Reference only	15%
#113	Qwen2 Instruct 72B qwen2-72b-instruct	Alibaba Cloud	—	Reference only	14.7%
#114	Phi-4 phi-4	Microsoft	open	Reference only	14.3%
#115	Gemma 3n E4B Instruct gemma-3n-e4b	Google	—	Reference only	13.7%
#116	Llama Nemotron Super 49B v1.5 (Non-reasoning) llama-nemotron-super-49b-v1-5	NVIDIA	—	Reference only	13.7%
#117	Grok 2 (Dec '24) grok-2-1212	xAI	—	Reference only	13.3%
#118	Llama 3.1 Tulu3 405B tulu3-405b	Allen Institute for AI	—	Reference only	13.3%
#119	Mistral Saba mistral-saba	Mistral AI	—	Reference only	13%
#120	Qwen2.5 Coder 32B Instruct qwen2-5-coder-32b-instruct	Alibaba Cloud	open	Reference only	12%
#121	Qwen2.5 Turbo qwen-turbo	Alibaba Cloud	closed	Reference only	12%
#122	GPT-4o (Aug '24) gpt-4o-2024-08-06	OpenAI	closed	Reference only	11.7%
#123	GPT-4o mini gpt-4o-mini	OpenAI	closed	Reference only	11.7%
#124	GPT-4o (May '24) gpt-4o-2024-05-13	OpenAI	closed	Reference only	11%
#125	Mistral Large 2 mistral-large-2	Mistral AI	closed	Reference only	11%
#126	Qwen2.5 Instruct 32B qwen2.5-32b-instruct	Alibaba Cloud	open	Reference only	11%
#127	Gemma 3n E4B Instruct Preview (May '25) gemma-3n-e4b-preview-0520	Google	—	Reference only	10.7%
#128	Nova Lite nova-lite	Amazon	—	Reference only	10.7%
#129	Nova Pro nova-pro	Amazon	closed	Reference only	10.7%
#130	GPT-4o (ChatGPT) gpt-4o-chatgpt	OpenAI	—	Reference only	10.3%
#131	Grok Beta grok-beta	xAI	—	Reference only	10.3%
#132	Qwen3 0.6B (Reasoning) qwen3-0.6b-instruct-reasoning	Alibaba Cloud	—	Reference only	10%
#133	Claude 3.5 Sonnet (June '24) claude-35-sonnet-june-24	Anthropic	—	Reference only	9.7%
#134	Command A+ command-a-plus	Cohere	open	Reference only	9.7%
#135	Qwen3 1.7B (Non-reasoning) qwen3-1.7b-instruct	Alibaba Cloud	—	Reference only	9.7%
#136	Gemini 1.5 Flash (May '24) gemini-1-5-flash-may-2024	Google	—	Reference only	9.3%
#137	Llama 3.2 Instruct 11B (Vision) llama-3-2-instruct-11b-vision	Meta	—	Reference only	9.3%
#138	Mistral Large 2 (Jul '24) mistral-large-2407	Mistral AI	—	Reference only	9.3%
#139	Mistral Small 3.1 mistral-small-3-1	Mistral AI	—	Reference only	9.3%
#140	Phi-4 Multimodal Instruct phi-4-multimodal	Microsoft	—	Reference only	9.3%
#141	Gemma 3n E2B Instruct gemma-3n-e2b	Google	—	Reference only	9%
#142	Gemini 1.5 Pro (May '24) gemini-1-5-pro-may-2024	Google	—	Reference only	8%
#143	Mistral Small 3 mistral-small-3	Mistral AI	—	Reference only	8%
#144	Nova Micro nova-micro	Amazon	—	Reference only	8%
#145	Llama 3.1 Instruct 8B llama-3-1-instruct-8b	Meta	—	Reference only	7.7%
#146	Pixtral Large pixtral-large-2411	Mistral AI	—	Reference only	7%
#147	Devstral Medium devstral-medium	Mistral AI	—	Reference only	6.7%
#148	Devstral Small (May '25) devstral-small-2505	Mistral AI	open	Reference only	6.7%
#149	Llama 3.2 Instruct 3B llama-3-2-instruct-3b	Meta	—	Reference only	6.7%
#150	Gemma 3 4B Instruct gemma-3-4b	Google	—	Reference only	6.3%
#151	Mistral Small (Sep '24) mistral-small	Mistral AI	—	Reference only	6.3%
#152	Jamba 1.7 Large jamba-1-7-large	AI21 Labs	—	Reference only	5.7%
#153	Qwen2.5 Coder Instruct 7B qwen2-5-coder-7b-instruct	Alibaba Cloud	—	Reference only	5.3%
#154	Llama 3.2 Instruct 90B (Vision) llama-3-2-instruct-90b-vision	Meta	—	Reference only	5%
#155	Claude 3 Sonnet claude-3-sonnet	Anthropic	—	Reference only	4.7%
#156	DeepHermes 3 - Mistral 24B Preview (Non-reasoning) deephermes-3-mistral-24b-preview	Nous Research	—	Reference only	4.7%
#157	Granite 3.3 8B (Non-reasoning) granite-3-3-8b-instruct	IBM	—	Reference only	4.7%
#158	Jamba 1.5 Large jamba-1-5-large	AI21 Labs	—	Reference only	4.7%
#159	Jamba 1.6 Large jamba-1-6-large	AI21 Labs	—	Reference only	4.7%
#160	Phi-3 Mini Instruct 3.8B phi-3-mini	Microsoft	—	Reference only	4%
#161	Mistral Medium mistral-medium	Mistral AI	—	Reference only	3.7%
#162	Claude 2.1 claude-21	Anthropic	—	Reference only	3.3%
#163	Claude 3 Opus claude-3-opus	Anthropic	closed	Reference only	3.3%
#164	Claude 3.5 Haiku claude-3-5-haiku	Anthropic	—	Reference only	3.3%
#165	Gemini 1.5 Flash-8B gemini-1-5-flash-8b	Google	—	Reference only	3.3%
#166	Jamba 1.6 Mini jamba-1-6-mini	AI21 Labs	—	Reference only	3.3%
#167	DBRX Instruct dbrx-instruct	Databricks	open	Reference only	3%
#168	Phi-4 Mini Instruct phi-4-mini	Microsoft	—	Reference only	3%
#169	Hermes 3 - Llama-3.1 70B hermes-3-llama-3-1-70b	Nous Research	—	Reference only	2.3%
#170	LFM 40B lfm-40b	LiquidAI	—	Reference only	2.3%
#171	Llama 2 Chat 13B llama-2-chat-13b	Meta	—	Reference only	1.7%
#172	Qwen3 0.6B (Non-reasoning) qwen3-0.6b-instruct	Alibaba Cloud	—	Reference only	1.7%
#173	Jamba 1.7 Mini jamba-1-7-mini	AI21 Labs	—	Reference only	1.3%
#174	Claude 3 Haiku claude-3-haiku	Anthropic	closed	Reference only	1%
#175	Jamba 1.5 Mini jamba-1-5-mini	AI21 Labs	—	Reference only	1%
#176	Command-R (Mar '24) command-r-03-2024	Cohere	—	Reference only	0.7%
#177	Command-R+ (Apr '24) command-r-plus-04-2024	Cohere	—	Reference only	0.7%
#178	Gemini 1.0 Pro gemini-1-0-pro	Google	closed	Reference only	0.7%
#179	Mistral Small (Feb '24) mistral-small-2402	Mistral AI	—	Reference only	0.7%
#180	Devstral Small (Jul '25) devstral-small	Mistral AI	—	Reference only	0.3%
#181	Claude 2.0 claude-2	Anthropic	—	Reference only	0%
#182	Claude Instant claude-instant	Anthropic	—	Reference only	0%
#183	DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning) deephermes-3-llama-3-1-8b-preview	Nous Research	—	Reference only	0%
#184	Gemma 3 1B Instruct gemma-3-1b	Google	—	Reference only	0%
#185	Llama 2 Chat 70B llama-2-chat-70b	Meta	—	Reference only	0%
#186	Llama 2 Chat 7B llama-2-chat-7b	Meta	—	Reference only	0%
#187	Llama 3 Instruct 70B llama-3-instruct-70b	Meta	—	Reference only	0%
#188	Llama 3 Instruct 8B llama-3-instruct-8b	Meta	—	Reference only	0%
#189	Llama 3.2 Instruct 1B llama-3-2-instruct-1b	Meta	—	Reference only	0%
#190	Mistral 7B Instruct mistral-7b-instruct	Mistral AI	—	Reference only	0%
#191	Mistral Large (Feb '24) mistral-large	Mistral AI	—	Reference only	0%
#192	Mixtral 8x22B Instruct mistral-8x22b-instruct	Mistral AI	—	Reference only	0%
#193	Mixtral 8x7B Instruct mixtral-8x7b-instruct	Mistral AI	—	Reference only	0%
#194	OpenChat 3.5 (1210) openchat-35	OpenChat	—	Reference only	0%

Rank

Model

Provider

License

Evidence use

Score

GPT-5

gpt-5

OpenAI

closed

Reference only

95.7%

Grok 4

grok-4

xAI

closed

Reference only

94.3%

o4-mini

OpenAI

closed

Reference only

94%

Qwen3 235B A22B 2507 (Reasoning)

qwen3-235b-a22b-instruct-2507-reasoning

Alibaba Cloud

—

Reference only

94%

Grok 3 mini Reasoning (high)

grok-3-mini-reasoning

xAI

—

Reference only

93.3%

GPT-5 (medium)

gpt-5-medium

OpenAI

closed

Reference only

91.7%

Qwen3 30B A3B 2507 (Reasoning)

qwen3-30b-a3b-2507-reasoning

Alibaba Cloud

—

Reference only

90.7%

OpenAI

closed

Reference only

90.3%

DeepSeek-R1

deepseek-r1

DeepSeek

open

Reference only

89.3%

#10

Gemini 2.5 Pro

gemini-2.5-pro

Google

closed

Reference only

88.7%

#11

GLM-4.5 (Reasoning)

glm-4.5

Z.AI

open

Reference only

87.3%

#12

Gemini 2.5 Pro Preview (Mar' 25)

gemini-2-5-pro-03-25

Google

—

Reference only

87%

#13

Llama Nemotron Super 49B v1.5 (Reasoning)

llama-nemotron-super-49b-v1-5-reasoning

NVIDIA

—

Reference only

86%

#14

o3-mini (high)

o3-mini-high

OpenAI

—

Reference only

86%

#15

MiniMax M1 80k

minimax-m1-80k

MiniMax

closed

Reference only

84.7%

#16

EXAONE 4.0 32B (Reasoning)

exaone-4-0-32b-reasoning

LG AI Research

—

Reference only

84.3%

#17

Gemini 2.5 Flash Preview (Reasoning)

gemini-2-5-flash-reasoning-04-2025

Google

—

Reference only

84.3%

#18

Gemini 2.5 Pro Preview (May' 25)

gemini-2-5-pro-05-06

Google

—

Reference only

84.3%

#19

Qwen3 235B A22B (Reasoning)

qwen3-235b-a22b-instruct-reasoning

Alibaba Cloud

—

Reference only

84%

#20

GPT-5 (low)

gpt-5-low

OpenAI

—

Reference only

83%

#21

Gemini 2.5 Flash (Reasoning)

gemini-2-5-flash-reasoning

Google

—

Reference only

82.3%

#22

MiniMax M1 40k

minimax-m1-40k

MiniMax

—

Reference only

81.3%

#23

Qwen3 32B (Reasoning)

qwen3-32b-instruct-reasoning

Alibaba Cloud

—

Reference only

80.7%

#24

Sonar Reasoning Pro

sonar-reasoning-pro

Perplexity

closed

Reference only

79%

#25

QwQ 32B

qwq-32b

Alibaba Cloud

—

Reference only

78%

#26

Claude 4 Sonnet (Reasoning)

claude-4-sonnet-thinking

Anthropic

—

Reference only

77.3%

#27

o3-mini

OpenAI

closed

Reference only

77%

#28

Sonar Reasoning

sonar-reasoning

Perplexity

—

Reference only

77%

#29

Qwen3 14B (Reasoning)

qwen3-14b-instruct-reasoning

Alibaba Cloud

—

Reference only

76.3%

#30

Claude 4 Opus (Reasoning)

claude-4-opus-thinking

Anthropic

—

Reference only

75.7%

#31

Qwen3 30B A3B (Reasoning)

qwen3-30b-a3b-instruct-reasoning

Alibaba Cloud

—

Reference only

75.3%

#32

Llama 3.1 Nemotron Ultra 253B v1 (Reasoning)

llama-3-1-nemotron-ultra-253b-v1-reasoning

NVIDIA

—

Reference only

74.7%

#33

Qwen3 8B (Reasoning)

qwen3-8b-instruct-reasoning

Alibaba Cloud

—

Reference only

74.7%

#34

Qwen3 30B A3B 2507 Instruct

qwen3-30b-a3b-2507

Alibaba Cloud

—

Reference only

72.7%

#35

OpenAI

closed

Reference only

72.3%

#36

Qwen3 235B A22B 2507 Instruct

qwen3-235b-a22b-instruct-2507

Alibaba Cloud

—

Reference only

71.7%

#37

Magistral Small 1

magistral-small

Mistral AI

open

Reference only

71.3%

#38

Llama 3.1 Nemotron Nano 4B v1.1 (Reasoning)

llama-3-1-nemotron-nano-4b-reasoning

NVIDIA

—

Reference only

70.7%

#39

Gemini 2.5 Flash-Lite (Reasoning)

gemini-2-5-flash-lite-reasoning

Google

—

Reference only

70.3%

#40

Magistral Medium 1

magistral-medium

Mistral AI

—

Reference only

70%

#41

Kimi K2

kimi-k2

Kimi

—

Reference only

69.3%

#42

Solar Pro 2 (Reasoning)

solar-pro-2-reasoning

Upstage

—

Reference only

69%

#43

DeepSeek R1 Distill Qwen 32B

deepseek-r1-distill-qwen-32b

DeepSeek

open

Reference only

68.7%

#44

DeepSeek R1 (Jan '25)

deepseek-r1-0120

DeepSeek

—

Reference only

68.3%

#45

GLM-4.5-Air

glm-4-5-air

Z.AI

closed

Reference only

67.3%

#46

DeepSeek R1 Distill Llama 70B

deepseek-r1-distill-llama-70b

DeepSeek

—

Reference only

67%

#47

DeepSeek R1 Distill Qwen 14B

deepseek-r1-distill-qwen-14b

DeepSeek

—

Reference only

66.7%

#48

Solar Pro 2 (Preview) (Reasoning)

solar-pro-2-preview-reasoning

Upstage

—

Reference only

66.3%

#49

Qwen3 4B (Reasoning)

qwen3-4b-instruct-reasoning

Alibaba Cloud

—

Reference only

65.7%

#50

DeepSeek R1 0528 Qwen3 8B

deepseek-r1-qwen3-8b

DeepSeek

—

Reference only

65%

#51

o1-mini

OpenAI

—

Reference only

60.3%

#52

Llama 3.3 Nemotron Super 49B v1 (Reasoning)

llama-3-3-nemotron-super-49b-reasoning

NVIDIA

—

Reference only

58.3%

#53

Claude 4 Opus (Non-reasoning)

claude-4-opus

Anthropic

—

Reference only

56.3%

#54

DeepSeek V3 0324

deepseek-v3-0324

DeepSeek

—

Reference only

52%

#55

Qwen3 1.7B (Reasoning)

qwen3-1.7b-instruct-reasoning

Alibaba Cloud

—

Reference only

51%

#56

Reka Flash 3

reka-flash-3

Reka AI

—

Reference only

51%

#57

Gemini 2.0 Flash Thinking Experimental (Jan '25)

gemini-2-0-flash-thinking-exp-0121

Google

—

Reference only

50%

#58

Gemini 2.5 Flash

gemini-2.5-flash

Google

closed

Reference only

50%

#59

Gemini 2.5 Flash-Lite (Non-reasoning)

gemini-2-5-flash-lite

Google

—

Reference only

50%

#60

ERNIE 4.5 300B A47B

ernie-4-5-300b-a47b

Baidu

—

Reference only

49.3%

#61

Claude 3.7 Sonnet (Reasoning)

claude-3-7-sonnet-thinking

Anthropic

—

Reference only

48.7%

#62

Sonar

sonar

Perplexity

closed

Reference only

48.7%

#63

Qwen3 Coder 480B A35B Instruct

qwen3-coder-480b-a35b-instruct

Alibaba Cloud

open

Reference only

47.7%

#64

Exaone 4.0 32B

exaone-4-0-32b

LG AI Research

open

Reference only

47%

#65

QwQ 32B-Preview

QwQ-32B-Preview

Alibaba Cloud

—

Reference only

45.3%

#66

Mistral Medium 3

mistral-medium-3

Mistral AI

closed

Reference only

44%

#67

GPT-4.1

gpt-4-1

OpenAI

closed

Reference only

43.7%

#68

Gemini 2.5 Flash Preview (Non-reasoning)

gemini-2-5-flash-04-2025

Google

—

Reference only

43.3%

#69

GPT-4.1 mini

gpt-4-1-mini

OpenAI

closed

Reference only

43%

#70

Claude 4 Sonnet

claude-4-sonnet

Anthropic

closed

Reference only

40.7%

#71

Solar Pro 2

solar-pro-2

Upstage

closed

Reference only

40.7%

#72

Llama 4 Maverick

llama-4-maverick

Meta

open

Reference only

39%

#73

GPT-5 (minimal)

gpt-5-minimal

OpenAI

—

Reference only

36.7%

#74

Gemini 2.0 Pro Experimental (Feb '25)

gemini-2-0-pro-experimental-02-05

Google

—

Reference only

36%

#75

DeepSeek R1 Distill Llama 8B

deepseek-r1-distill-llama-8b

DeepSeek

—

Reference only

33.3%

#76

Gemini 2.0 Flash (Feb '25)

gemini-2-0-flash

Google

—

Reference only

33%

#77

Grok 3

grok-3

xAI

—

Reference only

33%

#78

GPT-4o (March 2025, chatgpt-4o-latest)

gpt-4o-chatgpt-03-25

OpenAI

—

Reference only

32.7%

#79

Qwen3 235B A22B (Non-reasoning)

qwen3-235b-a22b-instruct

Alibaba Cloud

—

Reference only

32.7%

#80

Mistral Small 3.2

mistral-small-3-2

Mistral AI

—

Reference only

32.3%

#81

Gemini 2.0 Flash-Lite (Preview)

gemini-2-0-flash-lite-preview

Google

—

Reference only

30.3%

#82

Qwen3 32B (Non-reasoning)

qwen3-32b-instruct

Alibaba Cloud

—

Reference only

30.3%

#83

Gemini 2.0 Flash (experimental)

gemini-2-0-flash-experimental

Google

—

Reference only

30%

#84

Llama 3.3 Instruct 70B

llama-3-3-instruct-70b

Meta

—

Reference only

30%

#85

Qwen3 Coder 30B A3B Instruct

qwen3-coder-30b-a3b-instruct

Alibaba Cloud

open

Reference only

29.7%

#86

Solar Pro 2 (Preview) (Non-reasoning)

solar-pro-2-preview

Upstage

—

Reference only

29.7%

#87

Sonar Pro

sonar-pro

Perplexity

closed

Reference only

29%

#88

Llama 4 Scout

llama-4-scout

Meta

open

Reference only

28.3%

#89

Qwen3 14B (Non-reasoning)

qwen3-14b-instruct

Alibaba Cloud

—

Reference only

28%

#90

Gemini 2.0 Flash-Lite (Feb '25)

gemini-2-0-flash-lite-001

Google

—

Reference only

27.7%

#91

Qwen3 30B A3B (Non-reasoning)

qwen3-30b-a3b-instruct

Alibaba Cloud

—

Reference only

26%

#92

DeepSeek V3

deepseek-v3

DeepSeek

open

Reference only

25.3%

#93

Gemma 3 27B

gemma-3-27b

Google

open

Reference only

25.3%

#94

Llama 3.1 Nemotron Instruct 70B

llama-3-1-nemotron-instruct-70b

NVIDIA

—

Reference only

24.7%

#95

Qwen3 8B (Non-reasoning)

qwen3-8b-instruct

Alibaba Cloud

—

Reference only

24.3%

#96

GPT-4.1 nano

gpt-4-1-nano

OpenAI

closed

Reference only

23.7%

#97

Qwen2.5 Max

qwen-2-5-max

Alibaba Cloud

—

Reference only

23.3%

#98

Gemini 1.5 Pro

gemini-1-5-pro

Google

closed

Reference only

23%

#99

Claude 3.7 Sonnet (Non-reasoning)

claude-3-7-sonnet

Anthropic

—

Reference only

22.3%

#100

Gemma 3 12B Instruct

gemma-3-12b

Google

—

Reference only

22%

#101

Llama 3.1 Instruct 405B

llama-3-1-instruct-405b

Meta

—

Reference only

21.3%

#102

Qwen3 4B (Non-reasoning)

qwen3-4b-instruct

Alibaba Cloud

—

Reference only

21.3%

#103

Sarvam M (Reasoning)

sarvam-m-reasoning

Sarvam

—

Reference only

20.3%

#104

Llama 3.3 Nemotron Super 49B v1 (Non-reasoning)

llama-3-3-nemotron-super-49b

NVIDIA

—

Reference only

19.3%

#105

Gemini 1.5 Flash (Sep '24)

gemini-1-5-flash

Google

—

Reference only

18%

#106

DeepSeek R1 Distill Qwen 1.5B

deepseek-r1-distill-qwen-1-5b

DeepSeek

—

Reference only

17.7%

#107

Llama 3.1 Instruct 70B

llama-3-1-instruct-70b

Meta

—

Reference only

17.3%

#108

Nova Premier

nova-premier

Amazon

—

Reference only

17%

#109

Qwen2.5 Instruct 72B

qwen2-5-72b-instruct

Alibaba Cloud

open

Reference only

16%

#110

Claude 3.5 Sonnet (Oct '24)

claude-35-sonnet

Anthropic

—

Reference only

15.7%

#111

GPT-4 Turbo

gpt-4-turbo

OpenAI

closed

Reference only

15%

#112

GPT-4o

gpt-4o

OpenAI

closed

Reference only

15%

#113

Qwen2 Instruct 72B

qwen2-72b-instruct

Alibaba Cloud

—

Reference only

14.7%

#114

Phi-4

phi-4

Microsoft

open

Reference only

14.3%

#115

Gemma 3n E4B Instruct

gemma-3n-e4b

Google

—

Reference only

13.7%

#116

Llama Nemotron Super 49B v1.5 (Non-reasoning)

llama-nemotron-super-49b-v1-5

NVIDIA

—

Reference only

13.7%

#117

Grok 2 (Dec '24)

grok-2-1212

xAI

—

Reference only

13.3%

#118

Llama 3.1 Tulu3 405B

tulu3-405b

Allen Institute for AI

—

Reference only

13.3%

#119

Mistral Saba

mistral-saba

Mistral AI

—

Reference only

13%

#120

Qwen2.5 Coder 32B Instruct

qwen2-5-coder-32b-instruct

Alibaba Cloud

open

Reference only

12%

#121

Qwen2.5 Turbo

qwen-turbo

Alibaba Cloud

closed

Reference only

12%

#122

GPT-4o (Aug '24)

gpt-4o-2024-08-06

OpenAI

closed

Reference only

11.7%

#123

GPT-4o mini

gpt-4o-mini

OpenAI

closed

Reference only

11.7%

#124

GPT-4o (May '24)

gpt-4o-2024-05-13

OpenAI

closed

Reference only

11%

#125

Mistral Large 2

mistral-large-2

Mistral AI

closed

Reference only

11%

#126

Qwen2.5 Instruct 32B

qwen2.5-32b-instruct

Alibaba Cloud

open

Reference only

11%

#127

Gemma 3n E4B Instruct Preview (May '25)

gemma-3n-e4b-preview-0520

Google

—

Reference only

10.7%

#128

Nova Lite

nova-lite

Amazon

—

Reference only

10.7%

#129

Nova Pro

nova-pro

Amazon

closed

Reference only

10.7%

#130

GPT-4o (ChatGPT)

gpt-4o-chatgpt

OpenAI

—

Reference only

10.3%

#131

Grok Beta

grok-beta

xAI

—

Reference only

10.3%

#132

Qwen3 0.6B (Reasoning)

qwen3-0.6b-instruct-reasoning

Alibaba Cloud

—

Reference only

10%

#133

Claude 3.5 Sonnet (June '24)

claude-35-sonnet-june-24

Anthropic

—

Reference only

9.7%

#134

Command A+

command-a-plus

Cohere

open

Reference only

9.7%

#135

Qwen3 1.7B (Non-reasoning)

qwen3-1.7b-instruct

Alibaba Cloud

—

Reference only

9.7%

#136

Gemini 1.5 Flash (May '24)

gemini-1-5-flash-may-2024

Google

—

Reference only

9.3%

#137

Llama 3.2 Instruct 11B (Vision)

llama-3-2-instruct-11b-vision

Meta

—

Reference only

9.3%

#138

Mistral Large 2 (Jul '24)

mistral-large-2407

Mistral AI

—

Reference only

9.3%

#139

Mistral Small 3.1

mistral-small-3-1

Mistral AI

—

Reference only

9.3%

#140

Phi-4 Multimodal Instruct

phi-4-multimodal

Microsoft

—

Reference only

9.3%

#141

Gemma 3n E2B Instruct

gemma-3n-e2b

Google

—

Reference only

#142

Gemini 1.5 Pro (May '24)

gemini-1-5-pro-may-2024

Google

—

Reference only

#143

Mistral Small 3

mistral-small-3

Mistral AI

—

Reference only

#144

Nova Micro

nova-micro

Amazon

—

Reference only

#145

Llama 3.1 Instruct 8B

llama-3-1-instruct-8b

Meta

—

Reference only

7.7%

#146

Pixtral Large

pixtral-large-2411

Mistral AI

—

Reference only

#147

Devstral Medium

devstral-medium

Mistral AI

—

Reference only

6.7%

#148

Devstral Small (May '25)

devstral-small-2505

Mistral AI

open

Reference only

6.7%

#149

Llama 3.2 Instruct 3B

llama-3-2-instruct-3b

Meta

—

Reference only

6.7%

#150

Gemma 3 4B Instruct

gemma-3-4b

Google

—

Reference only

6.3%

#151

Mistral Small (Sep '24)

mistral-small

Mistral AI

—

Reference only

6.3%

#152

Jamba 1.7 Large

jamba-1-7-large

AI21 Labs

—

Reference only

5.7%

#153

Qwen2.5 Coder Instruct 7B

qwen2-5-coder-7b-instruct

Alibaba Cloud

—

Reference only

5.3%

#154

Llama 3.2 Instruct 90B (Vision)

llama-3-2-instruct-90b-vision

Meta

—

Reference only

#155

Claude 3 Sonnet

claude-3-sonnet

Anthropic

—

Reference only

4.7%

#156

DeepHermes 3 - Mistral 24B Preview (Non-reasoning)

deephermes-3-mistral-24b-preview

Nous Research

—

Reference only

4.7%

#157

Granite 3.3 8B (Non-reasoning)

granite-3-3-8b-instruct

IBM

—

Reference only

4.7%

#158

Jamba 1.5 Large

jamba-1-5-large

AI21 Labs

—

Reference only

4.7%

#159

Jamba 1.6 Large

jamba-1-6-large

AI21 Labs

—

Reference only

4.7%

#160

Phi-3 Mini Instruct 3.8B

phi-3-mini

Microsoft

—

Reference only

#161

Mistral Medium

mistral-medium

Mistral AI

—

Reference only

3.7%

#162

Claude 2.1

claude-21

Anthropic

—

Reference only

3.3%

#163

Claude 3 Opus

claude-3-opus

Anthropic

closed

Reference only

3.3%

#164

Claude 3.5 Haiku

claude-3-5-haiku

Anthropic

—

Reference only

3.3%

#165

Gemini 1.5 Flash-8B

gemini-1-5-flash-8b

Google

—

Reference only

3.3%

#166

Jamba 1.6 Mini

jamba-1-6-mini

AI21 Labs

—

Reference only

3.3%

#167

DBRX Instruct

dbrx-instruct

Databricks

open

Reference only

#168

Phi-4 Mini Instruct

phi-4-mini

Microsoft

—

Reference only

#169

Hermes 3 - Llama-3.1 70B

hermes-3-llama-3-1-70b

Nous Research

—

Reference only

2.3%

#170

LFM 40B

lfm-40b

LiquidAI

—

Reference only

2.3%

#171

Llama 2 Chat 13B

llama-2-chat-13b

Meta

—

Reference only

1.7%

#172

Qwen3 0.6B (Non-reasoning)

qwen3-0.6b-instruct

Alibaba Cloud

—

Reference only

1.7%

#173

Jamba 1.7 Mini

jamba-1-7-mini

AI21 Labs

—

Reference only

1.3%

#174

Claude 3 Haiku

claude-3-haiku

Anthropic

closed

Reference only

#175

Jamba 1.5 Mini

jamba-1-5-mini

AI21 Labs

—

Reference only

#176

Command-R (Mar '24)

command-r-03-2024

Cohere

—

Reference only

0.7%

#177

Command-R+ (Apr '24)

command-r-plus-04-2024

Cohere

—

Reference only

0.7%

#178

Gemini 1.0 Pro

gemini-1-0-pro

Google

closed

Reference only

0.7%

#179

Mistral Small (Feb '24)

mistral-small-2402

Mistral AI

—

Reference only

0.7%

#180

Devstral Small (Jul '25)

devstral-small

Mistral AI

—

Reference only

0.3%

#181

Claude 2.0

claude-2

Anthropic

—

Reference only

#182

Claude Instant

claude-instant

Anthropic

—

Reference only

#183

DeepHermes 3 - Llama-3.1 8B Preview (Non-reasoning)

deephermes-3-llama-3-1-8b-preview

Nous Research

—

Reference only

#184

Gemma 3 1B Instruct

gemma-3-1b

Google

—

Reference only

#185

Llama 2 Chat 70B

llama-2-chat-70b

Meta

—

Reference only

#186

Llama 2 Chat 7B

llama-2-chat-7b

Meta

—

Reference only

#187

Llama 3 Instruct 70B

llama-3-instruct-70b

Meta

—

Reference only

#188

Llama 3 Instruct 8B

llama-3-instruct-8b

Meta

—

Reference only

#189

Llama 3.2 Instruct 1B

llama-3-2-instruct-1b

Meta

—

Reference only

#190

Mistral 7B Instruct

mistral-7b-instruct

Mistral AI

—

Reference only

#191

Mistral Large (Feb '24)

mistral-large

Mistral AI

—

Reference only

#192

Mixtral 8x22B Instruct

mistral-8x22b-instruct

Mistral AI

—

Reference only

#193

Mixtral 8x7B Instruct

mixtral-8x7b-instruct

Mistral AI

—

Reference only

#194

OpenChat 3.5 (1210)

openchat-35

OpenChat

—

Reference only

Definition and scoring

Organisation

Mathematical Association of America

What does American Invitational Mathematics Examination 2024 measure?

The 2024 edition of AIME, maintaining the same format of 15 challenging mathematics problems with integer answers from 000 to 999.

Which model scores highest on American Invitational Mathematics Examination 2024?

GPT-5 by OpenAI currently leads with 95.7%.

How many models are evaluated on American Invitational Mathematics Examination 2024?

194 models in the LuminaBench cohort have a qualifying score on this benchmark.

Does this affect overall Lumina rank?

No. This benchmark is display-only and does not enter the overall Lumina composite.

Search LUMINA BENCH

Navigation

American Invitational Mathematics Examination 2024

Leaderboard · 194 models

GPT-5

Grok 4

o4-mini