About ProgramBench hidden-test pass rate after episode 1
Definition and scoring
- Organisation
- Yang et al.
- Category
- Coding
- Version
- 2026
- Direction
- higher is better
- Ranking use
- Reference
- Contamination risk
- Unknown
Program-reconstruction hidden-test pass rate after the first of five sequential long-context episodes. Every genuine source row stays tied to its exact model label, configuration, benchmark version and evaluation system. The summary chart shows one best compatible score per canonical product; Score 2.0 admits only explicitly mapped, frozen protocols and keeps incompatible configurations separate.
Open benchmark source ↗