The LLM benchmark aggregator.

NVIDIA Nemotron 3 Ultra 550B A55B

NVIDIA · 561B · A55B · open weights

Strongest in Instruction following (#15 of 62), weakest in Safety (#250 of 327). Above par in 14 of 33 scopes. Among the models it meets almost everywhere, it finishes behind Claude Fable 5 and Gemini 3 Pro and ahead of GPT-5 nano and DeepSeek V3.2.

§ 1 · Profile

What it is good at

Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.

Knowledge52.4−10.1#82/1971/3

Academic knowledge55.7−8.2#54/1231/1

Factuality53.1−7.7—/0✱✱

Reasoning50−15.4#100/1953/5

Science57−6.5#44/1221/1

Reasoning45.4−23.6#86/1471/3

Mathematics49.3−14.5#89/1521/2

Expert reasoning49.5−16.9—/0✱✱

Human preference59.2−9.6#102/3541/1

Professional48.7−13.7#114/1941/2

Finance51.1−11.3#89/1621/1

Medical46.5−20#107/1491/1

Legal46.5−19.4#114/1731/1

Agents50.8−15.5#119/2982/6

Knowledge work51.2−19.5#91/2082/2

Tool use34.2−39.8—/81✱0/1

Web research36.8−26.8—/36✱0/2

Workflow automation30.7−35.3—/0✱✱

Core abilities47.2−18.4#124/2002/4

Instruction following59.6−14.2#15/621/1

Language40.4−26.2#56/621/1

General intelligence42.7−26.3#138/1801/3

Long context53.2−11.4—/0✱✱

Coding38.8−27.3#173/1792/6

Agentic coding38.3−28.4#168/1732/5

Code generation52.8−12.5—/20✱0/1

Safety46.9−14.5#250/3271/3

Secure code63.1−3.1#27/2681/1

Toxicity avoidance52.3−6.7#165/2661/1

Fairness43.9−26.1#215/2901/2

Jailbreak resistance32.9−32.1#240/2661/1

Harm refusal44.8−16.7#242/2901/2

§ 2 · Head to head

What it beats, and what beats it

The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for NVIDIA Nemotron 3 Ultra 550B A55B, left for the other.

Llama 4 Scout Instructwon 17 · lost 5

GPT-4o miniwon 17 · lost 6

GPT-4.1 nanowon 17 · lost 7

Command Awon 15 · lost 7

GPT-5.4 nanowon 14 · lost 7

Claude 3.5 Haikuwon 13 · lost 7

Gemini 2.0 Flashwon 14 · lost 8

DeepSeek R1won 14 · lost 8

O3 Miniwon 13 · lost 8

Grok 4.1 Fastwon 13 · lost 8

GPT OSS 20Bwon 12 · lost 8

Grok 3won 13 · lost 9

Llama 4 Maverick Instructwon 14 · lost 10

GPT OSS 120Bwon 14 · lost 10

DeepSeek V3won 14 · lost 10

GPT-5.4 miniwon 11 · lost 9

Gemma 4 31Bwon 11 · lost 9

Claude Haiku 4.5won 12 · lost 10

Claude Sonnet 4won 13 · lost 11

DeepSeek V3.2won 13 · lost 11

GPT-5 nanowon 11 · lost 10

GPT-4owon 11 · lost 11

GPT-4.1 miniwon 12 · lost 12

Kimi K2won 11 · lost 12

Claude 3.7 Sonnetwon 10 · lost 11

MiMo V2.5 Prowon 10 · lost 11

Grok 4.3won 11 · lost 13

O4 Miniwon 10 · lost 13

Gemini 2.5 Flashwon 10 · lost 13

GPT-4.1won 10 · lost 13

MiniMax M2.7won 9 · lost 12

GLM-4.6won 9 · lost 12

Grok 4won 9 · lost 13

O1won 8 · lost 12

Mistral Large 4won 9 · lost 14

GPT-5 miniwon 9 · lost 15

Qwen3 235B A22B Instructwon 8 · lost 15

Inklingwon 8 · lost 18

Qwen3.8 27Bwon 6 · lost 14

Gemini 2.5 Prowon 7 · lost 17

Kimi K2.6won 7 · lost 17

MiniMax M3won 7 · lost 19

Kimi K2 Thinkingwon 5 · lost 15

GPT-6 Lunawon 6 · lost 18

Gemini 3.5 Flashwon 6 · lost 19

GLM-5.3 Flashwon 6 · lost 20

Claude Opus 4won 5 · lost 17

O3won 5 · lost 18

DeepSeek V4 Flashwon 5 · lost 21

GPT-5.6 Lunawon 5 · lost 21

GLM-5.2won 5 · lost 21

GPT-6 Solwon 4 · lost 19

Claude Sonnet 4.6won 4 · lost 19

Kimi K2.5won 4 · lost 20

Gemini 3.6 Flashwon 4 · lost 20

MiMo V2.6 Flashwon 3 · lost 17

MiMo V2.6 Prowon 3 · lost 18

Claude Opus 4.6won 3 · lost 19

Claude Sonnet 4.5won 3 · lost 21

Muse Spark 1.1won 3 · lost 21

Claude Sonnet 5won 3 · lost 22

DeepSeek V4 Prowon 3 · lost 23

Grok 4.5won 3 · lost 23

GPT-5.6 Solwon 3 · lost 23

Claude Opus 4.5won 2 · lost 18

GLM-5.3won 2 · lost 18

Gemini 3.1 Prowon 2 · lost 18

GPT-5.1won 2 · lost 20

GLM-5.1won 2 · lost 20

Qwen3.7 Maxwon 2 · lost 23

GPT-5.6 Terrawon 2 · lost 24

GPT-5.5won 2 · lost 24

Claude Opus 4.8won 2 · lost 24

Gemini 3.8 Flashwon 2 · lost 24

Claude Opus 4.7won 2 · lost 24

Kimi K3won 1 · lost 19

Claude Fable 5.1won 1 · lost 19

Grok 4.7won 1 · lost 22

GPT-5won 1 · lost 23

Claude Opus 5.5won 1 · lost 23

GPT-5.2won 1 · lost 24

Claude Opus 5won 1 · lost 24

Grok 4.6won 1 · lost 25

GPT-5.4won 1 · lost 25

Qwen3.8 Maxwon 1 · lost 25

Gemini 3 Prowon 0 · lost 20

Claude Fable 5won 0 · lost 20

§ 3 · Sources

Where the numbers come from

10 publications, 52 figures. Every one links to the page it was read from.

LMArena Text ↗1 measureread 2026-11-11

LMArena Text 1426

Artificial Analysis ↗1 measureread 2026-11-11

Artificial Analysis Intelligence Index 12

GDPval-AA ↗1 measureread 2026-11-11

GDPval-AA 1021

AA-Briefcase ↗1 measureread 2026-11-11

AA-Briefcase 877

LiveBench 1.0 ↗8 measuresread 2026-11-11

LiveBench 1.0 67.4LiveBench 1.0 · Reasoning 74.7LiveBench 1.0 · Coding 70.7LiveBench 1.0 · Agentic Coding 38.7LiveBench 1.0 · Mathematics 88.7LiveBench 1.0 · Data Analysis 54.5LiveBench 1.0 · Language 70.8LiveBench 1.0 · Instruction Following 73.4

SimpleBench ↗1 measureread 2026-11-11

SimpleBench 41.7%

Vals.ai ↗11 measuresread 2026-11-11

Vals · Legal Research Bench 15.38%Vals · LegalBench 82.07%Vals · Finance Agent 37.67%Vals · CorpFin 65.46%Vals · TaxEval 73.1%Vals · MedCode 38.62%Vals · SWE-bench Verified 69% (Mini-SWE-agent)Vals · Vibe Code Bench 7.64% (OpenHands)Vals · Code Migration 4.91%Vals · GPQA Diamond 86.11%Vals · MMLU Pro 85.76%

Enkrypt AI Safety Leaderboard ↗6 measuresread 2026-11-11

Enkrypt · Jailbreak risk 27.2%Enkrypt · Harmful content risk 9.4%Enkrypt · CBRN risk 52.7%Enkrypt · Toxicity risk 4.6%Enkrypt · Bias risk 87.3%Enkrypt · Insecure code risk 6.2%

✱✱ IFM — K2 Horizon launch ↗the model’s own publisher17 measuresread 2026-11-03

GDPVal-AA 1162tau3-Banking 14.2%Toolathlon Verified 34.3%Automation Bench Public 8%Apex-Agents (pass@1) 9%MCPMark 45.7%BrowseComp 44.4%WildClawBench 34.2%Terminal-Bench 2.1 53.9%SciCode 39.9%SWE Bench Pro 38.7%Humanity's Last Exam (without tools) 28.4%GPQA Diamond 86.7%CritPt 3.1%AA-LCR 71%AA-Omniscience Accuracy 23%AA-Omniscience Non-Hallucination 70%

✱ CellCog — K2 Horizon analysis ↗5 measuresread 2026-11-11

GDPVal-AA (Elo) 1162Toolathlon Verified 34.3%Terminal-Bench 2.1 53.9%SWE-bench Pro (strict) 38.7%MCPMark 45.7%

Badge

[![PublicAI Index](https://publicai.io/model-index/badge?model=nvidia-nemotron-3-ultra-550b-a55b)](https://publicai.io/model-index/m/nvidia-nemotron-3-ultra-550b-a55b)