MBPP Benchmark

MBPP Benchmark | Phaseo

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
IBM	Granite 4.1 8B	29 Apr 2026	87.30	pass@1	Yes	Source
IBM	Granite 4.1 30B	29 Apr 2026	85.45	pass@1	Yes	Source
IBM	Granite 4.1 3B	29 Apr 2026	71.16	pass@1	Yes	Source
Nvidia	Llama 3.3 Nemotron Super 49B v1	18 Mar 2025	0.91	-	Yes	Source
Nvidia	Llama 3.3 Nemotron Super 49B V1.5	-	0.91	inferred version-family alias from llama-3.3-nemotron-super-49b-v1	Yes	Source
Qwen	Qwen 72B	-	0.88	inferred family alias from qwen-2.5-72b-instruct (score=0.3060; benches=14)	Yes	Source
Nvidia	Llama 3.1 Nemotron Nano 4B V1.1	-	0.85	inferred high-confidence family alias from llama-3.1-nemotron-nano-8b-v1 (score=0.5523; benches=7)	Yes	Source
Nvidia	Llama 3.1 Nemotron Nano 8B V1	18 Mar 2025	0.85	-	Yes	Source
Qwen	Qwen 2.5 Coder 32B Instruct	-	0.84	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4641; benches=28)	Yes	Source
Qwen	Qwen 2.5 VL 32B Instruct	24 Mar 2025	0.84	-	Yes	Source
Qwen	Qwen 2.5 VL 3B Instruct	-	0.84	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4914; benches=28)	Yes	Source
Qwen	Qwen 14B	-	0.82	inferred family alias from qwen-2.5-14b-instruct (score=0.3060; benches=16)	Yes	Source
Qwen	Qwen 3 235B A22B	-	0.81	-	Yes	Source
Microsoft	Phi 3.5 MoE instruct	23 Aug 2024	0.81	-	Yes	Source
Qwen	Qwen 2 Math 72B	-	0.80	inferred high-confidence family alias from qwen2-72b-instruct (score=0.4667; benches=17)	Yes	Source
Qwen	Qwen 2 Math RM 72B	-	0.80	inferred family alias from qwen2-72b-instruct (score=0.3917; benches=17)	Yes	Source
Qwen	Qwen 7B	-	0.79	inferred family alias from qwen-2.5-7b-instruct (score=0.3083; benches=14)	Yes	Source
Mistral	Codestral (2025-07-30)	30 Jul 2025	0.78	inferred version-family alias from codestral-22b	Yes	Source
Mistral	Codestral (2024-05-29)	29 May 2024	0.78	inferred modality/version alias from codestral-22b	Yes	Source
Mistral	Codestral (2025-01-13)	13 Jan 2025	0.78	inferred modality/version alias from codestral-22b	Yes	Source
Meta	Llama 4 Maverick	05 Apr 2025	0.78	-	Yes	Source
Google	Gemini Diffusion	20 May 2025	0.76	-	Yes	Source
Qwen	Qwen 2.5 Coder 3B	-	0.73	inferred family alias from qwen2.5-omni-7b (score=0.3000; benches=45)	Yes	Source
Qwen	Qwen 2.5 Coder 7B	-	0.73	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4700; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B	-	0.73	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4767; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B PRM800K	-	0.73	inferred family alias from qwen2.5-omni-7b (score=0.3696; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math PRM 7B	-	0.73	inferred family alias from qwen2.5-omni-7b (score=0.4092; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 3B	-	0.73	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4933; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 7B	-	0.73	-	Yes	Source
Microsoft	Phi 3 Mini 128K Instruct	-	0.70	inferred family alias from phi-3.5-mini-instruct (score=0.3533; benches=31)	Yes	Source
Microsoft	Phi 3.5 mini instruct	23 Aug 2024	0.70	-	Yes	Source
Meta	Llama 4 Scout	05 Apr 2025	0.68	-	Yes	Source
Qwen	Qwen 2 Audio 7B	-	0.67	inferred modality/version alias from qwen2-7b-instruct	Yes	Source
Qwen	Qwen 2 Math 7B	-	0.67	inferred high-confidence family alias from qwen2-7b-instruct (score=0.4706; benches=14)	Yes	Source

Average Score

Score Range

Leading Model

MBPP

Recorded Results

Average Score

Score Range

Leading Model

Models Using This Benchmark