Tau2 Airline Benchmark | Phaseo

Tau2 Airline Benchmark | Phaseo

Models Using This Benchmark

Organisation	Model	Reported	Top Score	Info	Self Reported	Source
Arcee AI	Trinity Large Thinking	01 Apr 2026	88%	Hugging Face model card benchmark table (arcee-ai/Trinity-Large-Thinking)	Yes	Source
OpenAI	GPT 5.1 Chat	13 Nov 2025	67%	inferred alias from gpt-5.1-2025-11-13	Yes	Source
Anthropic	Claude Haiku 4.5	15 Oct 2025	63.60%	inferred alias from claude-haiku-4-5-20251001	Yes	Source
Qwen	Qwen 3 Next 80B A3B Thinking	-	60.50%	-	Yes	Source
Meituan	LongCat Flash Chat	-	58%	inferred high-confidence family alias from longcat-flash-chat (score=0.4667; benches=16)	Yes	Source
Qwen	Qwen 3 235B A22B Thinking 2507	-	58%	-	Yes	Source
Nvidia	Nemotron 3 Super	11 Mar 2026	56.25%	-	Yes	Source
Inception	Mercury 2	24 Feb 2026	53%	-	Yes	Source
Nvidia	Nemotron Nano 3 30B A3B	15 Dec 2025	48%	-	Yes	Source
OpenAI	GPT 4o (2024-08-06)	06 Aug 2024	45.50%	-	Yes	Source
OpenAI	GPT 4o Audio (2025-06-03)	03 Jun 2025	45.50%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-10-01)	01 Oct 2024	45.50%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Audio (2024-12-17)	17 Dec 2024	45.50%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2025-06-03)	03 Jun 2025	45.50%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Realtime Preview (2024-10-01)	01 Oct 2024	45.50%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Search Preview	11 Mar 2025	45.50%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe	20 Mar 2025	45.50%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
OpenAI	GPT 4o Transcribe Diarize	15 Oct 2025	45.50%	inferred modality/version alias from gpt-4o-2024-08-06	Yes	Source
Qwen	Qwen 3 Next 80B A3B Instruct	-	45.50%	-	Yes	Source