Knowledge · Benchmark profile

Massive Multitask Language Understanding Professional

An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.

Reference2024Active43 models

Data verified 27 Jul 2026 · Methodology 1.8.0

Benchmark score on Massive Multitask Language Understanding Professional

Leaderboard · 43 models

Qwen3.7-Max leads at 89.6%, followed by Claude Opus 4.5 (89.5%) and Qwen3.6 Plus (88.5%).

Display onlyKnowledge2024

#1Closed

Qwen3.7-Max

Alibaba Cloud

qwen3.7-max

89.6%

Overall #11 · Context 1M tokens

#2Closed

Claude Opus 4.5

Anthropic

claude-opus-4-5

89.5%

Overall #49 · Context 1M tokens

#3Closed

Qwen3.6 Plus

Alibaba Cloud

qwen3-6-plus

88.5%

Overall #29 · Context 1M tokens

View

ModelsColours

Alibaba CloudAnthropicDeepSeekMoonshot AINVIDIA

Lumina

Massive Multitask Language Understanding Professional leaderboard

10 ranked models · higher is better · labels show rank and score

View accessible chart data

Accessible data for Massive Multitask Language Understanding Professional leaderboard
Model	Rank	Provider	Score
Qwen3.7-Max	#1	Alibaba Cloud	89.6%
Claude Opus 4.5	#2	Anthropic	89.5%
Qwen3.6 Plus	#3	Alibaba Cloud	88.5%
Qwen3.7-Plus	#4	Alibaba Cloud	88.5%
Qwen3.5 397B	#5	Alibaba Cloud	87.8%
DeepSeek V4 Pro (Max)	#6	DeepSeek	87.5%
DeepSeek V4 Pro (High)	#7	DeepSeek	87.1%
Kimi K2.5	#8	Moonshot AI	87.1%
Kimi K2.5 (Reasoning)	#9	Moonshot AI	87.1%
Nemotron 3 Ultra	#10	NVIDIA	86.8%

Benchmark score table (43 models)

One best score per model · higher is better

Massive Multitask Language Understanding Professional leaderboard ranked by score
Rank	Model	Provider	License	Evidence use	Score
#1	Qwen3.7-Max qwen3.7-max	Alibaba Cloud	closed	Reference only	89.6%
#2	Claude Opus 4.5 claude-opus-4-5	Anthropic	closed	Reference only	89.5%
#3	Qwen3.6 Plus qwen3-6-plus	Alibaba Cloud	closed	Reference only	88.5%
#4	Qwen3.7-Plus qwen3.7-plus	Alibaba Cloud	closed	Reference only	88.5%
#5	Qwen3.5 397B qwen3.5-397b-a17b	Alibaba Cloud	open	Estimated reference	87.8%
#6	DeepSeek V4 Pro (Max) deepseek-v4-pro-max	DeepSeek	open	Estimated reference	87.5%
#7	DeepSeek V4 Pro (High) deepseek-v4-pro-high	DeepSeek	open	Estimated reference	87.1%
#8	Kimi K2.5 kimi-k2.5	Moonshot AI	open	Reference only	87.1%
#9	Kimi K2.5 (Reasoning) kimi-k2-5-reasoning	Moonshot AI	closed	Estimated reference	87.1%
#10	Nemotron 3 Ultra nemotron-3-ultra-550b	NVIDIA	open	Estimated reference	86.8%
#11	Qwen3.5-122B-A10B qwen3-5-122b-a10b	Alibaba Cloud	open	Reference only	86.7%
#12	DeepSeek V4 Flash (High) deepseek-v4-flash-high	DeepSeek	open	Estimated reference	86.4%
#13	DeepSeek V4 Flash (Max) deepseek-v4-flash-max	DeepSeek	open	Estimated reference	86.2%
#14	Qwen3.6 27B qwen3.6-27b	Alibaba Cloud	open	Estimated reference	86.2%
#15	Qwen3.5 27B qwen3.5-27b	Alibaba Cloud	open	Reference only	86.1%
#16	GLM-5 glm-5	Z.AI	open	Reference only	85.7%
#17	Qwen3.5-35B-A3B qwen3-5-35b-a3b	Alibaba Cloud	open	Reference only	85.3%
#18	Gemma 4 31B gemma-4-31b	Google	open	Reference only	85.2%
#19	Qwen3.6-35B-A3B qwen3-6-35b-a3b	Alibaba Cloud	open	Estimated reference	85.2%
#20	MAI-Thinking-1 mai-thinking-1	Microsoft	closed	Estimated reference	85%
#21	MiMo-V2-Flash mimo-v2-flash	Xiaomi	open	Reference only	84.9%
#22	GLM-4.7 glm-4.7	Z.AI	open	Reference only	84.3%
#23	DeepSeek V4 Flash deepseek-v4-flash	DeepSeek	open	Estimated reference	83%
#24	Qwen3 235B 2507 qwen3-235b-2507	Alibaba Cloud	open	Estimated reference	83%
#25	DeepSeek V4 Pro deepseek-v4-pro	DeepSeek	open	Reference only	82.9%
#26	Gemma 4 26B A4B gemma-4-26b-a4b	Google	open	Reference only	82.6%
#27	Claude Opus 4.6 claude-opus-4-6	Anthropic	closed	Reference only	82%
#28	Exaone 4.0 32B exaone-4-0-32b	LG AI Research	open	Estimated reference	81.8%
#29	Claude Sonnet 4.6 claude-sonnet-4-6	Anthropic	closed	Reference only	79.2%
#30	Nemotron 3 Nano Omni 30B A3B nemotron-3-nano-omni-30b-a3b	NVIDIA	open	Estimated reference	77.3%
#31	Gemma 4 12B gemma-4-12b	Google	open	Estimated reference	77.2%
#32	Celeris-1 celeris-1	Celeris	closed	Estimated reference	75.9%
#33	DeepSeek V3 deepseek-v3	DeepSeek	open	Reference only	75.9%
#34	ZAYA1-8B zaya1-8b	Zyphra	open	Estimated reference	74.2%
#35	DeepSeek V4 Pro Base deepseek-v4-pro-base	DeepSeek	open	Estimated reference	73.5%
#36	Gemma 4 E4B gemma-4-e4b	Google	open	Estimated reference	69.4%
#37	DeepSeek V4 Flash Base deepseek-v4-flash-base	DeepSeek	open	Estimated reference	68.3%
#38	ZAYA1-74B-Preview zaya1-74b-preview	Zyphra	open	Estimated reference	68.1%
#39	Gemma 4 E2B gemma-4-e2b	Google	open	Estimated reference	60%
#40	Soofi S 30B-A3B soofi-s-30b-a3b	Soofi Project	open	Estimated reference	51.4%
#41	MiniCPM5-1B minicpm5-1b	OpenBMB	open	Estimated reference	48.9%
#42	LFM2.5-230M lfm2-5-230m	LiquidAI	open	Estimated reference	20.3%
#43	LFM2.5-VL-450M lfm2-5-vl-450m	LiquidAI	open	Estimated reference	19.3%

The top of this snapshot is led by Qwen3.7-Max at 89.6%; third place is 1.1 points behind. The top-10 spread is 2.8 points.

Compare top models

Qwen3.7-Max vs Claude Opus 4.5 Claude Opus 4.5 vs Qwen3.6 Plus Qwen3.6 Plus vs Qwen3.7-Plus Qwen3.7-Plus vs Qwen3.5 397B

About Massive Multitask Language Understanding Professional

Definition and scoring

Organisation: Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, Wenhu Chen
Category: Knowledge
Version: 2024
Direction: higher is better
Ranking use: Reference
Contamination risk: Unknown

An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models. Results stay tied to the exact model variant and evaluation system. Multiple systems for the same model use the best published score on this page; overall Lumina scoring uses the median of ranking-eligible rows.

Open benchmark source ↗

FAQ

Quick answers

What does Massive Multitask Language Understanding Professional measure?

An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.

Which model scores highest on Massive Multitask Language Understanding Professional?

Qwen3.7-Max by Alibaba Cloud currently leads with 89.6%.

How many models are evaluated on Massive Multitask Language Understanding Professional?

43 models in the LuminaBench cohort have a qualifying score on this benchmark.

Does this affect overall Lumina rank?

No. This benchmark is display-only and does not enter the overall Lumina composite.

Artificial Analysis Openness Index

2026 · 21 results · Reference

Artificial Analysis MMLU-Pro

2026 · 8 results · Reference

Massive Multitask Language Understanding

2020 · 16 results · Reference

Graduate-Level Google-Proof Q&A

2023 · 140 results · Reference

SuperGPQA: Scaling LLM Evaluation Across 285 Graduate Disciplines

2025 · 38 results · Reference

AGIEval

2026 · 6 results · Reference

Knowledge · Benchmark profile

Massive Multitask Language Understanding Professional

An enhanced version of MMLU with 10 answer choices instead of 4, featuring more reasoning-focused questions that better differentiate frontier models.

Reference2024Active43 models

Data verified 27 Jul 2026 · Methodology 1.8.0

Benchmark score on Massive Multitask Language Understanding Professional