MMMU

Overview

Type: percentage

View benchmark source

Recorded Results

128

Average Score

67.34%

Score Range

0% - 86%

Leading Model (lowest score)

0% - GPT 3.5 Turbo 16K 0613

Scores Over Time

Individual benchmark scores plotted by date.

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
OpenAI	GPT 3.5 Turbo 16K 0613	13 Jun 2023	0%	inferred family alias from gpt-3.5-turbo-0125 (score=0.4244; benches=8)	No	Source
OpenAI	GPT 3.5 Turbo (2023-11-06)	28 Sept 2023	0%	inferred modality/version alias from gpt-3.5-turbo-0125	No	Source
OpenAI	GPT 3.5 Turbo Instruct	06 Nov 2023	0%	inferred high-confidence family alias from gpt-3.5-turbo-0125 (score=0.5539; benches=8)	No	Source
DeepSeek	DeepSeek VL2 Tiny	13 Dec 2024	40.70%	-	Yes	Source
Microsoft	Phi 3 Vision 128K Instruct	-	43%	inferred family alias from phi-3.5-vision-instruct (score=0.3495; benches=9)	Yes	Source
Microsoft	Phi 3.5 vision instruct	23 Aug 2024	43%	-	Yes	Source
Google	Gemini 1.0 Nano	06 Dec 2023	47.90%	inferred modality/version alias from gemini-1.0-pro	No	Source
Google	Gemini 1.0 Pro	06 Dec 2023	47.90%	-	No	Source
Google	Gemini 1.0 Pro Vision 001	15 Feb 2024	47.90%	inferred modality/version alias from gemini-1.0-pro	No	Source
DeepSeek	DeepSeek VL2 Small	13 Dec 2024	48%	-	Yes	Source
Meta	Llama 3.2 11B Vision Instruct	-	50.70%	inferred high-confidence family alias from llama-3.2-11b-instruct (score=0.5873; benches=11)	Yes	Source
Meta	Llama 3.2 1B Instruct	25 Sept 2024	50.70%	inferred family alias from llama-3.2-11b-instruct (score=0.4200; benches=11)	Yes	Source
DeepSeek	DeepSeek VL2	13 Dec 2024	51.10%	-	Yes	Source
Mistral	Pixtral 12B	17 Sept 2024	52.50%	inferred version-family alias from pixtral-12b-2409	Yes	Source
SpaceXAI	Grok 1.5	28 Mar 2024	53.60%	-	Yes	Source
SpaceXAI	Grok 1.5V	12 Apr 2024	53.60%	-	Yes	Source
Google	Gemini 1.5 Flash 8B	15 Mar 2024	53.70%	-	Yes	Source
Google	Gemini 1.5 Flash 8B Exp (2024-08-27)	27 Aug 2024	53.70%	inferred alias from gemini-1.5-flash-8b	Yes	Source
Google	Gemini 1.5 Flash 8B Exp (2024-09-24)	24 Sept 2024	53.70%	inferred alias from gemini-1.5-flash-8b	Yes	Source
Microsoft	Phi 4 multimodal instruct	01 Feb 2025	55.10%	-	Yes	Source
Qwen	Qwen 2.5 0.5B	-	58.60%	inferred family alias from qwen2.5-vl-7b (score=0.3131; benches=32)	Yes	Source
Qwen	Qwen 2.5 1.5B	-	58.60%	inferred family alias from qwen2.5-vl-7b (score=0.3131; benches=32)	Yes	Source
Qwen	Qwen 2.5 3B	-	58.60%	inferred family alias from qwen2.5-vl-7b (score=0.3531; benches=32)	Yes	Source
Qwen	Qwen 2.5 VL 7B Instruct	-	58.60%	-	Yes	Source
Qwen	Qwen 2.5 Coder 3B	-	59.20%	inferred family alias from qwen2.5-omni-7b (score=0.3000; benches=45)	Yes	Source
Qwen	Qwen 2.5 Coder 7B	-	59.20%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4700; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B	-	59.20%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4767; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math 7B PRM800K	-	59.20%	inferred family alias from qwen2.5-omni-7b (score=0.3696; benches=45)	Yes	Source
Qwen	Qwen 2.5 Math PRM 7B	-	59.20%	inferred family alias from qwen2.5-omni-7b (score=0.4092; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 3B	-	59.20%	inferred high-confidence family alias from qwen2.5-omni-7b (score=0.4933; benches=45)	Yes	Source
Qwen	Qwen 2.5 Omni 7B	-	59.20%	-	Yes	Source
OpenAI	GPT 4o Mini Audio Preview	17 Dec 2024	59.40%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini Realtime Preview	17 Dec 2024	59.40%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini Search Preview	11 Mar 2025	59.40%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini Transcribe (2025-12-15)	15 Dec 2025	59.40%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini Transcribe (2025-03-20)	20 Mar 2025	59.40%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini TTS (2025-12-15)	15 Dec 2025	59.40%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
OpenAI	GPT 4o Mini TTS (2025-03-20)	20 Mar 2025	59.40%	inferred modality/version alias from gpt-4o-mini-2024-07-18	Yes	Source
Meta	Llama 3.2 90B Vision Instruct	-	60.30%	inferred high-confidence family alias from llama-3.2-90b-instruct (score=0.5873; benches=13)	Yes	Source
Google	Gemini 1.5 Flash 001	23 May 2024	62.30%	inferred alias from gemini-1.5-flash	Yes	Source
Google	Gemini 1.5 Flash Preview	14 May 2024	62.30%	inferred alias from gemini-1.5-flash	Yes	Source
Mistral	Mistral Small 3.2	20 Jun 2025	62.50%	-	Yes	Source
SpaceXAI	Grok 2 Mini	13 Aug 2024	63.20%	-	Yes	Source
Mistral	Pixtral Large	18 Nov 2024	64%	-	Yes	Source
Google	Gemini 1.5 Pro 001	23 May 2024	65.90%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini 1.5 Pro Exp (2024-08-01)	01 Aug 2024	65.90%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini 1.5 Pro Exp (2024-08-27)	27 Aug 2024	65.90%	inferred alias from gemini-1.5-pro	Yes	Source
Google	Gemini Robotics ER 1.5 Preview	25 Sept 2025	65.90%	inferred family alias from gemini-1.5-pro (score=0.3717; benches=23)	Yes	Source
Google	LearnLM 1.5 Pro Experimental	19 Nov 2024	65.90%	inferred family alias from gemini-1.5-pro (score=0.3700; benches=23)	Yes	Source
Mistral	Magistral Small 1.0	10 Jun 2025	66%	-	Yes	Source
SpaceXAI	Grok 2	13 Aug 2024	66.10%	-	Yes	Source
Google	Gemini 2.0 Flash Lite	05 Feb 2025	68%	-	Yes	Source
Meta	Llama 4 Scout	05 Apr 2025	69.40%	-	Yes	Source
SpaceXAI	Grok 3 Mini Beta	19 Feb 2025	69.40%	-	Yes	Source
Mistral	Magistral Medium 1.0	10 Jun 2025	70%	-	Yes	Source
Moonshot	Kimi K1.5	20 Jan 2025	70%	-	Yes	Source
Qwen	Qwen 2.5 Coder 32B Instruct	-	70%	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4641; benches=28)	Yes	Source
Qwen	Qwen 2.5 VL 32B Instruct	24 Mar 2025	70%	-	Yes	Source
Qwen	Qwen 2.5 VL 3B Instruct	-	70%	inferred high-confidence family alias from qwen2.5-vl-32b (score=0.4914; benches=28)	Yes	Source
Qwen	Qwen 2.5 Math 72B	-	70.20%	inferred high-confidence family alias from qwen2.5-vl-72b (score=0.4700; benches=30)	Yes	Source
Qwen	Qwen 2.5 Math PRM 72B	-	70.20%	inferred family alias from qwen2.5-vl-72b (score=0.4050; benches=30)	Yes	Source
Qwen	Qwen 2.5 Math RM 72B	-	70.20%	inferred family alias from qwen2.5-vl-72b (score=0.4092; benches=30)	Yes	Source
Qwen	Qwen 2.5 VL 72B Instruct	26 Jan 2025	70.20%	-	Yes	Source
Qwen	QvQ 72B Preview	-	70.30%	-	Yes	Source
Google	Gemini 2.0 Flash Exp	-	70.70%	inferred alias from gemini-2.0-flash	Yes	Source
Google	Gemini 2.0 Pro Exp (2025-02-05)	05 Feb 2025	70.70%	inferred modality/version alias from gemini-2.0-flash	Yes	Source
Google	LearnLM 2.0 Flash Experimental	17 Apr 2025	70.70%	inferred family alias from gemini-2.0-flash (score=0.3700; benches=13)	Yes	Source
OpenAI	GPT 4o (2024-08-06)	06 Aug 2024	72.20%	-	Yes	Source
OpenAI	GPT 4o Audio (2025-06-03)	03 Jun 2025	72.20%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-10-01)	01 Oct 2024	72.20%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-12-17)	17 Dec 2024	72.20%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2025-06-03)	03 Jun 2025	72.20%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2024-10-01)	01 Oct 2024	72.20%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Search Preview	11 Mar 2025	72.20%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe	20 Mar 2025	72.20%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe Diarize	15 Oct 2025	72.20%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
Google	Gemini 2.5 Flash Lite Preview (2025-06-17)	17 Jun 2025	72.90%	No Thinking	Yes	Source
Google	Gemini 2.5 Flash Lite Preview (2025-09-25)	25 Sept 2025	72.90%	inferred alias from gemini-2.5-flash-lite	Yes	Source
Meta	Llama 4 Maverick	05 Apr 2025	73.40%	-	Yes	Source
Cohere	Command A+	20 May 2026	75.10%	-	Yes	Source
OpenAI	GPT 4.5	27 Feb 2025	75.20%	-	Yes	Source
Google	Gemini 2.0 Flash Thinking Exp (2024-12-19)	19 Dec 2024	75.40%	inferred alias from gemini-2.0-flash-thinking	Yes	Source
Google	Gemini 2.0 Flash Thinking Exp (2025-01-21)	21 Jan 2025	75.40%	inferred alias from gemini-2.0-flash-thinking	Yes	Source
OpenAI	GPT 5 Nano	07 Aug 2025	75.60%	High Reasoning Effort	Yes	Source
Google	Gemini 2.0 Flash	05 Feb 2025	76.70%	Pass@1	Yes	Source
Google	Gemini 2.5 Flash Preview (2025-04-17)	17 Apr 2025	76.70%	Pass@1	Yes	Source
Anthropic	Claude Opus 4.1	05 Aug 2025	77.10%	-	Yes	Source
SpaceXAI	Grok 3	18 Apr 2025	78%	-	Yes	Source
SpaceXAI	Grok 3 Beta	19 Feb 2025	78%	Think, Cons@64	Yes	Source
Google	Gemini 2.5 Computer Use Preview	07 Oct 2025	79.60%	inferred family alias from gemini-2.5-pro (score=0.3960; benches=16)	Yes	Source
Google	Gemini 2.5 Pro Experimental (2025-03-25)	25 Mar 2025	79.60%	inferred alias from gemini-2.5-pro	Yes	Source
Google	Gemini 2.5 Pro Preview (2025-05-06)	06 May 2025	79.60%	Pass@1	Yes	Source
Google	Gemini 2.5 Pro Preview TTS (2025-12-10)	10 Dec 2025	79.60%	inferred modality/version alias from gemini-2.5-pro	Yes	Source
Google	Gemini Embedding 2 Preview	10 Mar 2026	79.60%	manual fallback alias from gemini-2.5-pro	Yes	Source
ByteDance	Seed 2.0 Mini	14 Feb 2026	79.70%	Seed2 official benchmark table \| MMMU	Yes	Source
Google	Gemini 2.5 Flash Exp Native Audio Thinking Dialog	-	79.70%	inferred modality/version alias from gemini-2.5-flash	Yes	Source
Google	Gemini 2.5 Flash Image (Nano Banana)	02 Oct 2025	79.70%	inferred modality/version alias from gemini-2.5-flash	Yes	Source
Google	Gemini 2.5 Flash Image Preview (Nano Banana)	25 Aug 2025	79.70%	inferred modality/version alias from gemini-2.5-flash	Yes	Source
Google	Gemini 2.5 Flash Native Audio Preview (2025-09-23)	-	79.70%	inferred modality/version alias from gemini-2.5-flash	Yes	Source
Google	Gemini 2.5 Flash Preview (2025-05-20)	20 May 2025	79.70%	Pass@1	Yes	Source
Google	Gemini 2.5 Flash Preview (2025-09-25)	25 Sept 2025	79.70%	inferred alias from gemini-2.5-flash	Yes	Source
Google	Gemini 2.5 Flash Preview Native Audio Dialog	-	79.70%	inferred modality/version alias from gemini-2.5-flash	Yes	Source
Google	Gemini 2.5 Flash Preview TTS (2025-05-20)	20 May 2025	79.70%	inferred modality/version alias from gemini-2.5-flash	Yes	Source
Google	Gemini 2.5 Flash Preview TTS (2025-12-10)	10 Dec 2025	79.70%	inferred modality/version alias from gemini-2.5-flash	Yes	Source
Google	Gemini Live 2.5 Flash Preview	09 Apr 2025	79.70%	inferred high-confidence family alias from gemini-2.5-flash (score=0.5083; benches=14)	Yes	Source
Anthropic	Claude Opus 4.5	24 Nov 2025	80.72%	Avg@5, 64k Thinking	Yes	Source
Qwen	Qwen 3.5 35B A3B	24 Feb 2026	81.40%	-	Yes	Source
OpenAI	GPT 5 Mini	07 Aug 2025	81.60%	High Reasoning Effort	Yes	Source
OpenAI	o4 Mini	16 Apr 2025	81.60%	-	Yes	Source
OpenAI	o4 mini Deep Research	26 Jun 2025	81.60%	inferred modality/version alias from o4-mini	Yes	Source
Google	Gemini 2.5 Pro Preview (2025-06-05)	05 Jun 2025	82%	Single Attempt	Yes	Source
Google	Gemini 2.5 Pro Preview TTS (2025-05-20)	20 May 2025	82%	inferred family alias from gemini-2.5-pro-preview-06-05 (score=0.4243; benches=13)	Yes	Source
Qwen	Qwen 3.5 27B	24 Feb 2026	82.30%	-	Yes	Source
Qwen	Qwen 3.5 Flash	23 Feb 2026	82.30%	inferred family alias from qwen3.5-27b (score=0.4147; benches=81)	Yes	Source
ByteDance	Seed 2.0 Lite	14 Feb 2026	83.70%	Seed2 official benchmark table \| MMMU	Yes	Source
Qwen	Qwen 3.5 122B A10B	24 Feb 2026	83.90%	-	Yes	Source
OpenAI	GPT 5	07 Aug 2025	84.20%	With Thinking, Pass @ 1	Yes	Source
Qwen	Qwen 3.5 397B A17B	16 Feb 2026	85%	-	Yes	Source
ByteDance	Seed 2.0 Pro	14 Feb 2026	85.40%	Seed2 official benchmark table \| MMMU	Yes	Source
OpenAI	GPT 5.1 Chat	13 Nov 2025	85.40%	inferred alias from gpt-5.1-2025-11-13	Yes	Source
Qwen	Qwen 3.6 Plus	01 Apr 2026	86%	-	Yes	Source