MMLU Pro

Overview

Type: percentage

Recorded Results

Average Score

50.61%

Score Range

18.90% - 77.70%

Leading Model (lowest score)

18.90% - Qwen 3.5 0.8B

Scores Over Time

Individual benchmark scores plotted by date.

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
Qwen	Qwen 3.5 0.8B	02 Mar 2026	18.90%	-	Yes	Source
Qwen	Qwen 3.5 2B	02 Mar 2026	33.70%	-	Yes	Source
Nvidia	Nemotron Nano 3 30B A3B	15 Dec 2025	38.50%	-	Yes	Source
OpenAI	o3 mini	30 Jan 2025	39.90%	-	Yes	Source
OpenAI	GPT 4o (2024-08-06)	06 Aug 2024	40.30%	-	Yes	Source
OpenAI	GPT 4o Audio (2025-06-03)	03 Jun 2025	40.30%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-10-01)	01 Oct 2024	40.30%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-12-17)	17 Dec 2024	40.30%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2025-06-03)	03 Jun 2025	40.30%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2024-10-01)	01 Oct 2024	40.30%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Search Preview	11 Mar 2025	40.30%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe	20 Mar 2025	40.30%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe Diarize	15 Oct 2025	40.30%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	o4 Mini	16 Apr 2025	43%	-	Yes	Source
OpenAI	o4 mini Deep Research	26 Jun 2025	43%	inferred modality/version alias from o4-mini	Yes	Source
OpenAI	GPT 4.5	27 Feb 2025	43.80%	-	Yes	Source
MiniMax	Minimax M1 40K	16 Jun 2025	44.70%	-	Yes	-
MiniMax	MiniMax M1 80K	16 Jun 2025	44.70%	-	Yes	-
Qwen	Qwen 3.5 4B	02 Mar 2026	49%	-	Yes	Source
Moonshot	Kimi K2 (2025-09-05)	05 Sept 2025	51.10%	Acc	Yes	Source
Qwen	Qwen 3.5 9B	02 Mar 2026	54.50%	-	Yes	Source
OpenAI	GPT 5 Nano	07 Aug 2025	54.90%	High Reasoning Effort	Yes	Source
Nvidia	Nemotron 3 Super	11 Mar 2026	55.23%	-	Yes	Source
Qwen	Qwen 3.5 35B A3B	24 Feb 2026	60%	-	Yes	Source
Qwen	Qwen 3.5 27B	24 Feb 2026	60.80%	-	Yes	Source
Qwen	Qwen 3.5 Flash	23 Feb 2026	60.80%	inferred family alias from qwen3.5-27b (score=0.4147; benches=81)	Yes	Source
ByteDance	Seed 2.0 Mini	14 Feb 2026	61.10%	Seed2 official benchmark table \| MultiChallenge	Yes	Source
Qwen	Qwen 3.5 122B A10B	24 Feb 2026	61.50%	-	Yes	Source
OpenAI	GPT 5 Mini	07 Aug 2025	62.30%	High Reasoning Effort	Yes	Source
ByteDance	Seed 2.0 Lite	14 Feb 2026	63.20%	Seed2 official benchmark table \| MultiChallenge	Yes	Source
Qwen	Qwen 3.5 397B A17B	16 Feb 2026	67.60%	-	Yes	Source
ByteDance	Seed 2.0 Pro	14 Feb 2026	68.30%	Seed2 official benchmark table \| MultiChallenge	Yes	Source
OpenAI	GPT 5	07 Aug 2025	69.60%	With Thinking, Pass @ 1	Yes	Source
Amazon	Nova 2 Lite	02 Dec 2025	76.60%	-	Yes	Source
Amazon	Nova 2 Pro	02 Dec 2025	77.70%	-	Yes	Source