MMMU Pro

Overview

Type: percentage

Recorded Results

109

Average Score

62.58%

Score Range

33% - 83.20%

Leading Model (lowest score)

33% - Llama 3.2 11B Vision Instruct

Scores Over Time

Individual benchmark scores plotted by date.

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
Meta	Llama 3.2 11B Vision Instruct	-	33%	inferred high-confidence family alias from llama-3.2-11b-instruct (score=0.5873; benches=11)	Yes	Source
Meta	Llama 3.2 1B Instruct	25 Sept 2024	33%	inferred family alias from llama-3.2-11b-instruct (score=0.4200; benches=11)	Yes	Source
Qwen	Qwen 2.5 Coder 3B	-	36.60%	inferred family alias from qwen2.5-omni-7b (score=0.3000; benches=45)	Yes	Source
Qwen	Qwen 2.5 Coder 7B	-	36.60%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4700; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B	-	36.60%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4767; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B PRM800K	-	36.60%	inferred family alias from qwen2.5-omni-7b (score=0.3696; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math PRM 7B	-	36.60%	inferred family alias from qwen2.5-omni-7b (score=0.4092; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 3B	-	36.60%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4933; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 7B	-	36.60%	-	Yes	Source
Qwen	Qwen 2.5 0.5B	-	38.30%	inferred family alias from qwen2.5-vl-7b (score=0.3131; benches=32)	Yes	Source
Qwen	Qwen 2.5 1.5B	-	38.30%	inferred family alias from qwen2.5-vl-7b (score=0.3131; benches=32)	Yes	Source
Qwen	Qwen 2.5 3B	-	38.30%	inferred family alias from qwen2.5-vl-7b (score=0.3531; benches=32)	Yes	Source
Qwen	Qwen 2.5 VL 7B Instruct	-	38.30%	-	Yes	Source
Microsoft	Phi 4 multimodal instruct	01 Feb 2025	38.50%	-	Yes	Source
Meta	Llama 3.2 90B Vision Instruct	-	45.20%	inferred high-confidence family alias from llama-3.2-90b-instruct (score=0.5873; benches=13)	Yes	Source
Qwen	Qwen 2 VL 2B	-	46.20%	inferred family alias from qwen2-vl-72b (score=0.4083; benches=15)	Yes	Source
Qwen	Qwen 2 VL 72B	-	46.20%	-	Yes	Source
Qwen	Qwen 2.5 Coder 32B Instruct	-	49.50%	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4641; benches=28)	Yes	Source
Qwen	Qwen 2.5 VL 32B Instruct	24 Mar 2025	49.50%	-	Yes	Source
Qwen	Qwen 2.5 VL 3B Instruct	-	49.50%	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4914; benches=28)	Yes	Source
Qwen	Qwen 2.5 Math 72B	-	51.10%	inferred high-confidence family alias from qwen2.5-vl-72b (score=0.4700; benches=30)	Yes	Source
Qwen	Qwen 2.5 Math PRM 72B	-	51.10%	inferred family alias from qwen2.5-vl-72b (score=0.4050; benches=30)	Yes	Source
Qwen	Qwen 2.5 Math RM 72B	-	51.10%	inferred family alias from qwen2.5-vl-72b (score=0.4092; benches=30)	Yes	Source
Qwen	Qwen 2.5 VL 72B Instruct	26 Jan 2025	51.10%	-	Yes	Source
Qwen	Qwen 3 VL 4B Instruct	-	53.20%	-	Yes	-
Google	DiffusionGemma 26B A4B IT	10 Jun 2026	54.30%	-	Yes	Source
Qwen	Qwen 3 VL 8B Instruct	-	55.90%	-	Yes	-
Qwen	Qwen 3 4B Instruct 2507	-	57%	inferred family alias from qwen3-vl-4b-thinking (score=0.3462; benches=48)	Yes	-
Qwen	Qwen 3 4B SafeRL	-	57%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 4B Thinking 2507	-	57%	inferred family alias from qwen3-vl-4b-thinking (score=0.3462; benches=48)	Yes	-
Qwen	Qwen 3 Embedding 4B	-	57%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 Guard Gen 4B	-	57%	inferred family alias from qwen3-vl-4b-thinking (score=0.3400; benches=48)	Yes	-
Qwen	Qwen 3 Guard Stream 4B	-	57%	inferred family alias from qwen3-vl-4b-thinking (score=0.3371; benches=48)	Yes	-
Qwen	Qwen 3 Reranker 4B	-	57%	inferred family alias from qwen3-vl-4b-thinking (score=0.3850; benches=48)	Yes	-
Qwen	Qwen 3 VL 4B Thinking	-	57%	-	Yes	-
Meta	Llama 4 Maverick	05 Apr 2025	59.60%	-	Yes	Source
OpenAI	GPT 4o (2024-08-06)	06 Aug 2024	59.90%	-	Yes	Source
OpenAI	GPT 4o Audio (2025-06-03)	03 Jun 2025	59.90%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-10-01)	01 Oct 2024	59.90%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-12-17)	17 Dec 2024	59.90%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2025-06-03)	03 Jun 2025	59.90%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2024-10-01)	01 Oct 2024	59.90%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Search Preview	11 Mar 2025	59.90%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe	20 Mar 2025	59.90%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe Diarize	15 Oct 2025	59.90%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
Mistral	Mistral Small 1.0	26 Feb 2024	60%	inferred family alias from mistral-small-latest (score=0.3650; benches=9)	Yes	Source
Mistral	Mistral Small 2.0	17 Sept 2024	60%	inferred family alias from mistral-small-latest (score=0.3650; benches=9)	Yes	Source
Mistral	Mistral Small 4	16 Mar 2026	60%	-	Yes	Source
Mistral	Mistral Small Creative	16 Dec 2025	60%	inferred family alias from mistral-small-latest (score=0.4273; benches=9)	Yes	Source
Qwen	Qwen 3 8B	-	60.40%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.4600; benches=50)	Yes	-
Qwen	Qwen 3 Embedding 8B	-	60.40%	inferred family alias from qwen3-vl-8b-thinking (score=0.3850; benches=50)	Yes	-
Qwen	Qwen 3 Guard Gen 8B	-	60.40%	inferred family alias from qwen3-vl-8b-thinking (score=0.3400; benches=50)	Yes	-
Qwen	Qwen 3 Guard Stream 8B	-	60.40%	inferred family alias from qwen3-vl-8b-thinking (score=0.3371; benches=50)	Yes	-
Qwen	Qwen 3 Reranker 8B	-	60.40%	inferred family alias from qwen3-vl-8b-thinking (score=0.3850; benches=50)	Yes	-
Qwen	Qwen 3 VL 30B A3B Instruct	-	60.40%	-	Yes	-
Qwen	Qwen 3 VL 8B Thinking	-	60.40%	-	Yes	-
Qwen	Qwen 3 VL Embedding 8B	-	60.40%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.5232; benches=50)	Yes	-
Qwen	Qwen 3 VL Reranker 8B	-	60.40%	inferred high-confidence family alias from qwen3-vl-8b-thinking (score=0.5275; benches=50)	Yes	-
Amazon	Nova 2 Lite	02 Dec 2025	61.80%	-	Yes	Source
OpenAI	GPT 5 Nano	07 Aug 2025	62.60%	High Reasoning Effort	Yes	Source
Cohere	Command A+	20 May 2026	63%	-	Yes	Source
Qwen	Qwen 3 VL 30B A3B Thinking	-	63%	-	Yes	-
Amazon	Nova 2 Pro	02 Dec 2025	63.50%	-	Yes	Source
Qwen	Qwen 3 VL 32B Instruct	-	65.30%	-	Yes	-
Qwen	Qwen 3 VL 235B A22B Instruct	-	68.10%	-	Yes	-
Qwen	Qwen 3 VL 32B Thinking	-	68.10%	-	Yes	-
Google	Gemma 4 12B	03 Jun 2026	69.10%	-	Yes	Source
Qwen	Qwen 3 VL 235B A22B Thinking	-	69.30%	-	Yes	-
OpenAI	GPT 5.4 Nano	17 Mar 2026	69.50%	With Python	Yes	Source
ByteDance	Seed 2.0 Mini	14 Feb 2026	71.40%	Seed2 official benchmark table \| MMMU-Pro	Yes	Source
Google	Gemma 4 26B A4B	02 Apr 2026	73.80%	-	Yes	Source
OpenAI	GPT 5 Mini	07 Aug 2025	74.10%	High Reasoning Effort	Yes	Source
Qwen	Qwen 3.5 27B	24 Feb 2026	75%	-	Yes	Source
Qwen	Qwen 3.5 Flash	23 Feb 2026	75%	inferred family alias from qwen3.5-27b (score=0.4147; benches=81)	Yes	Source
Qwen	Qwen 3.5 35B A3B	24 Feb 2026	75.10%	-	Yes	Source
Anthropic	Claude Sonnet 4.6	17 Feb 2026	75.60%	With tools	Yes	Source
ByteDance	Seed 2.0 Lite	14 Feb 2026	76%	Seed2 official benchmark table \| MMMU-Pro	Yes	Source
Google	Gemini 3.1 Flash Image Preview (Nano Banana 2)	26 Feb 2026	76.80%	inferred modality/version alias from gemini-3.1-flash-lite-preview	Yes	Source
Google	Gemini 3.1 Flash-Lite	07 May 2026	76.80%	-	Yes	Source
Google	Gemini 3.1 Flash Lite Preview	03 Mar 2026	76.80%	-	Yes	Source
Google	Veo 3.1 Lite Preview	31 Mar 2026	76.80%	manual fallback alias from gemini-3.1-flash-lite-preview	Yes	Source
Google	Gemma 4 31B	02 Apr 2026	76.90%	-	Yes	Source
Qwen	Qwen 3.5 122B A10B	24 Feb 2026	76.90%	-	Yes	Source
Anthropic	Claude Opus 4.6	05 Feb 2026	77.30%	With tools	Yes	Source
Xiaomi	MiMo V2.5	22 Apr 2026	77.90%	Image understanding benchmark from official MiMo-V2.5 model card image	Yes	Source
OpenAI	GPT 5.4 Mini	17 Mar 2026	78%	With Python	Yes	Source
ByteDance	Seed 2.0 Pro	14 Feb 2026	78.20%	Seed2 official benchmark table \| MMMU-Pro	Yes	Source
OpenAI	GPT 5	07 Aug 2025	78.40%	With Thinking, Pass @ 1	Yes	Source
Moonshot	Kimi K2.5	27 Jan 2026	78.50%	-	Yes	Source
Qwen	Qwen 3.6 Plus	01 Apr 2026	78.80%	-	Yes	Source
Qwen	Qwen 3.5 397B A17B	16 Feb 2026	79%	-	Yes	Source
OpenAI	GPT 5.2 Chat	11 Dec 2025	79.50%	inferred alias from gpt-5.2-2025-12-11	Yes	Source
Meta	Muse Spark	08 Apr 2026	80.40%	Multimodal understanding	Yes	Source
Google	Gemini 3.1 Pro Preview	19 Feb 2026	80.50%	Search + Python + Browse	Yes	-
Google	Gemini 3 Pro Image Preview (Nano Banana Pro)	20 Nov 2025	81%	inferred modality/version alias from gemini-3-pro-preview	Yes	Source
Google	Gemini 3 Pro Preview	18 Nov 2025	81%	-	Yes	Source
Google	Gemini 3 Flash Preview	17 Dec 2025	81.20%	-	Yes	Source
OpenAI	GPT 5 Pro	07 Aug 2025	81.20%	inferred family alias from gpt-5.4 (score=0.4083; benches=19)	Yes	Source
OpenAI	GPT 5 Search API	14 Oct 2025	81.20%	inferred family alias from gpt-5.4 (score=0.3050; benches=19)	Yes	Source
Moonshot	Kimi K3	16 Jul 2026	81.60%	Average of three runs; official protocol preserves input order and prepends images to text input.	Yes	Source
OpenAI	GPT 5.4	05 Mar 2026	82.10%	With tools	Yes	Source
OpenAI	GPT 5.5	23 Apr 2026	83.20%	With tools	Yes	Source