Winogrande Benchmark

Winogrande Benchmark | Phaseo

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
OpenAI	GPT 4 32K 0613	13 Jun 2023	87.50%	inferred high-confidence family alias from gpt-4-0613 (score=0.4899; benches=12)	Yes	Source
Qwen	Qwen 2 Math 72B	-	85.10%	inferred high-confidence family alias from qwen2-72b-instruct (score=0.4667; benches=17)	Yes	Source
Qwen	Qwen 2 Math RM 72B	-	85.10%	inferred family alias from qwen2-72b-instruct (score=0.3917; benches=17)	Yes	Source
Nvidia	Llama 3.1 Nemotron 70B Instruct	01 Oct 2024	84.53%	-	Yes	Source
Microsoft	Phi 3.5 MoE instruct	23 Aug 2024	81.30%	-	Yes	Source
Mistral	Mistral Nemo 12B	18 Jul 2024	76.80%	inferred family alias from mistral-nemo-instruct-2407 (score=0.3250; benches=8)	Yes	Source
Google	Gemma 3n E4B	25 Jun 2025	71.70%	-	Yes	Source
Microsoft	Phi 3 Mini 128K Instruct	-	68.50%	inferred family alias from phi-3.5-mini-instruct (score=0.3533; benches=31)	Yes	Source
Microsoft	Phi 3.5 mini instruct	23 Aug 2024	68.50%	-	Yes	Source
Microsoft	Phi 4 Mini	01 Feb 2025	67%	-	Yes	Source
Google	Gemma 3n E2B	25 Jun 2025	66.80%	-	Yes	Source
Baidu	Ernie 4.5 21B A3B	-	51.30%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 21B A3B Thinking	-	51.30%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 300B A47B	-	51.30%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 Turbo	-	51.30%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 VL 28B A3B	-	51.30%	inferred version-family alias from ernie-4.5	Yes	Source
Baidu	Ernie 4.5 VL 424B A47B	-	51.30%	inferred version-family alias from ernie-4.5	Yes	Source

Winogrande