Coding · Benchmark profile

VulcanBench v3

An open software-engineering benchmark built from real merged post-cutoff pull requests across Python, Rust, TypeScript, JavaScript, and Go repositories.

Reference2026Active5 models

Data verified 1 Aug 2026 · Methodology 2.3.0

Benchmark score on VulcanBench v3

Leaderboard · 5 models

Grok 4.5 leads at 91.3%, followed by Claude Fable 5 (87%) and GPT-5.6 Sol (87%).

Display onlyCoding2026

#1Closed

Grok 4.5

xAI

grok-4.5

91.3%

Overall #12 · Context 500K tokens

#2Closed

Claude Fable 5

Anthropic

claude-fable-5

87%

Overall #2 · Context 1M tokens

#3Closed

GPT-5.6 Sol

OpenAI

gpt-5.6-sol

87%

Overall #4 · Context 1.05M tokens

ViewModelsColours

xAIAnthropicOpenAIMoonshot AI

Lumina

VulcanBench v3 leaderboard

Accessible data for VulcanBench v3 leaderboard
Model	Rank	Model ID	Provider	Score	Unit
Grok 4.5	#1	grok-4-5	xAI	91.3	percent
Claude Fable 5	#2	claude-fable-5	Anthropic	87	percent
GPT-5.6 Sol	#3	gpt-5-6-sol	OpenAI	87	percent
Claude Haiku 4.5	#4	claude-haiku-4-5	Anthropic	78.26	percent
Kimi K3	#5	kimi-k3	Moonshot AI	73.91	percent

Benchmark score table (5 models)

One best compatible score per canonical product · higher is better

VulcanBench v3 leaderboard ranked by score
Rank	Model	Provider	License	Evidence use	Score
#1	Grok 4.5 grok-4.5	xAI	closed	Source carrier	91.3%
#2	Claude Fable 5 claude-fable-5	Anthropic	closed	Source carrier	87%
#3	GPT-5.6 Sol gpt-5.6-sol	OpenAI	closed	Source carrier	87%
#4	Claude Haiku 4.5 claude-haiku-4-5-20251001	Anthropic	closed	Source carrier	78.3%
#5	Kimi K3 kimi-k3	Moonshot AI	open	Source carrier	73.9%

The top of this snapshot is led by Grok 4.5 at 91.3%; third place is 4.3 points behind. The top-5 spread is 17.4 points.

Configuration-preserving evidencePublished source rows (11)Exact source labels, configurations, versions and dated observations. Same-snapshot canonical/configuration projections are shown once.

Every distinct dated source-published row for VulcanBench v3, including exact model configuration labels
Exact published model / configuration	Canonical product	Score	Version / system	Evidence	Source / observed
Exact BenchLM registry variant Grok 4.5; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	Grok 4.5	91.3%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 2026-08-01 ↗Observed 2026-08-01 · checked 2026-08-01
Exact BenchLM registry variant Claude Fable 5; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	Claude Fable 5	87%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 2026-08-01 ↗Observed 2026-08-01 · checked 2026-08-01
Exact BenchLM registry variant GPT-5.6 Sol; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	GPT-5.6 Sol	87%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 2026-08-01 ↗Observed 2026-08-01 · checked 2026-08-01
Exact BenchLM registry variant Claude Haiku 4.5; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	Claude Haiku 4.5	78.3%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 2026-08-01 ↗Observed 2026-08-01 · checked 2026-08-01
Exact BenchLM registry variant Kimi K3; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	Kimi K3	73.9%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 2026-08-01 ↗Observed 2026-08-01 · checked 2026-08-01
Exact BenchLM registry variant Grok 4.5; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	Grok 4.5	91.3%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 2026-07-27 ↗Observed 2026-07-27 · checked 2026-07-27
Exact BenchLM registry variant Claude Fable 5; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	Claude Fable 5	87%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 2026-07-27 ↗Observed 2026-07-27 · checked 2026-07-27
Exact BenchLM registry variant GPT-5.6 Sol; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	GPT-5.6 Sol	87%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 2026-07-27 ↗Observed 2026-07-27 · checked 2026-07-27
Exact BenchLM registry variant Grok 4.5; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	Grok 4.5	91.3%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 21 July 2026 ↗Observed 2026-07-21 · checked 2026-07-21
Exact BenchLM registry variant Claude Fable 5; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	Claude Fable 5	87%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 21 July 2026 ↗Observed 2026-07-21 · checked 2026-07-21
Exact BenchLM registry variant GPT-5.6 Sol; bulk export does not retain a complete upstream harness configuration. source label without registered configuration ID	GPT-5.6 Sol	87%	2026Source-native system	Source carrier · no additional scoring weight source-checked	BenchLM public datasets — 21 July 2026 ↗Observed 2026-07-21 · checked 2026-07-21

Compare top models

Grok 4.5 vs Claude Fable 5 Claude Fable 5 vs GPT-5.6 Sol GPT-5.6 Sol vs Claude Haiku 4.5 Claude Haiku 4.5 vs Kimi K3

About VulcanBench v3

Definition and scoring

Organisation: VulcanBench contributors
Category: Coding
Version: 2026
Direction: higher is better
Ranking use: Reference
Contamination risk: Unknown

An open software-engineering benchmark built from real merged post-cutoff pull requests across Python, Rust, TypeScript, JavaScript, and Go repositories. Every genuine source row stays tied to its exact model label, configuration, benchmark version and evaluation system. The summary chart shows one best compatible score per canonical product; Score 2.0 admits only explicitly mapped, frozen protocols and keeps incompatible configurations separate.

Open benchmark source ↗

FAQ

Quick answers

What does VulcanBench v3 measure?

An open software-engineering benchmark built from real merged post-cutoff pull requests across Python, Rust, TypeScript, JavaScript, and Go repositories.

Which model scores highest on VulcanBench v3?

Grok 4.5 by xAI currently leads with 91.3%.

How many models are evaluated on VulcanBench v3?

5 canonical model products have a qualifying score, backed by 11 distinct dated source rows.

Does this affect overall Lumina rank?

No. This benchmark is display-only and does not enter the overall Lumina composite.

Coding · Benchmark profile

VulcanBench v3

An open software-engineering benchmark built from real merged post-cutoff pull requests across Python, Rust, TypeScript, JavaScript, and Go repositories.

Reference2026Active5 models

Data verified 1 Aug 2026 · Methodology 2.3.0

Benchmark score on VulcanBench v3