Reasoning · Benchmark profile
A suite of 23 challenging tasks from the BIG-Bench collaborative benchmark where prior language models failed to exceed average human performance, even with chain-of-thought prompting.
Data verified 27 Jul 2026 · Methodology 1.8.0
Benchmark score on BIG-Bench Hard
DeepSeek V4 Pro Base leads at 87.5%, followed by DeepSeek V4 Flash Base (86.9%) and Soofi S 30B-A3B (78.8%).
5 ranked models · higher is better · labels show rank and score
| Model | Rank | Provider | Score |
|---|---|---|---|
| DeepSeek V4 Pro Base | #1 | DeepSeek | 87.5% |
| DeepSeek V4 Flash Base | #2 | DeepSeek | 86.9% |
| Soofi S 30B-A3B | #3 | Soofi Project | 78.8% |
| MiniCPM5-1B | #4 | OpenBMB | 71.9% |
| Gemma 4 12B | #5 | 53% |
One best score per model · higher is better
| Rank | Model | Provider | License | Evidence use | Score |
|---|---|---|---|---|---|
| #1 | DeepSeek V4 Pro Base deepseek-v4-pro-base | DeepSeek | open | Estimated reference | 87.5% |
| #2 | DeepSeek V4 Flash Base deepseek-v4-flash-base | DeepSeek | open | Estimated reference | 86.9% |
| #3 | Soofi S 30B-A3B soofi-s-30b-a3b | Soofi Project | open | Estimated reference | 78.8% |
| #4 | MiniCPM5-1B minicpm5-1b | OpenBMB | open | Estimated reference | 71.9% |
| #5 | Gemma 4 12B gemma-4-12b | open | Estimated reference | 53% |
The top of this snapshot is led by DeepSeek V4 Pro Base at 87.5%; third place is 8.7 points behind. The top-5 spread is 34.5 points.
About BIG-Bench Hard
A suite of 23 challenging tasks from the BIG-Bench collaborative benchmark where prior language models failed to exceed average human performance, even with chain-of-thought prompting. Results stay tied to the exact model variant and evaluation system. Multiple systems for the same model use the best published score on this page; overall Lumina scoring uses the median of ranking-eligible rows.
Open benchmark source ↗FAQ
A suite of 23 challenging tasks from the BIG-Bench collaborative benchmark where prior language models failed to exceed average human performance, even with chain-of-thought prompting.
DeepSeek V4 Pro Base by DeepSeek currently leads with 87.5%.
5 models in the LuminaBench cohort have a qualifying score on this benchmark.
Related
