According to NVIDIA, AgentX says GB300 NVL72 lifts agentic AI throughput per megawatt
According to NVIDIA, AgentX evaluates agentic inference with long-context, KV-cache reuse, tool-call gaps, and MoE concurrency, then compares GB300 NVL72 with H200 NVL8 using tokens per megawatt and interactivity metrics.
What happened
According to NVIDIA, AgentX is meant to reflect agentic inference rather than simple chat: it targets long-context prefill, KV-cache reuse, interactive decode, tool-call gaps, and distributed MoE execution under realistic concurrency. NVIDIA says the benchmark tracks tokens per megawatt against user-experience measures including E2E normalized interactivity, standard interactivity, E2E latency, and time to first token. In the reported results, GB300 NVL72 delivers up to 15x higher AI-factory throughput per megawatt than H200 NVL8 on the AgentX workload and up to 10x lower token cost. NVIDIA also says GB300 NVL72 reaches roughly 80x throughput per megawatt versus H200 NVL8 on Kimi K3 2.8T at comparable interactivity. These are NVIDIA-reported benchmark claims, not independent reproduction.
Why it matters
FACT: According to NVIDIA, AgentX is designed for agentic inference patterns such as long-context prefill, KV-cache reuse, interactive decode, tool-call gaps, and MoE concurrency, and it judges systems with tokens per megawatt plus interactivity measures. INFERENCE: That makes the benchmark relevant for buyers deciding whether power, not just raw token rate, should anchor capacity planning for agent-serving infrastructure.
Engineering impact
FACT: NVIDIA reports that GB300 NVL72 reaches up to 15x higher AI-factory throughput per megawatt than H200 NVL8 on AgentX, and roughly 80x higher throughput per megawatt on Kimi K3 2.8T at comparable interactivity. INFERENCE: Engineering teams may need to test whether their own serving stack is constrained more by power efficiency and interactivity than by raw compute alone.
Leadership impact
FACT: NVIDIA frames AgentX around user-experience metrics such as E2E normalized interactivity, standard interactivity, E2E latency, and TTFT. INFERENCE: Leadership should treat this as a procurement and architecture signal for agentic workloads, but not as proof of universal superiority, because the reported gains are vendor-measured and workload-specific.
Editorial judgment
This changes an infrastructure decision: teams sizing agentic serving stacks should evaluate power-normalized throughput and interactivity, not just raw token rate. The tradeoff remains vendor-reported and workload-specific, so it is a procurement and benchmarking signal rather than proof of universal serving advantage.
Watch next
FACT: The benchmark description says AgentX varies concurrency to map throughput against interactivity and reports sustained throughput per provisioned megawatt. INFERENCE: The next question is whether independent runs with the same workload settings reproduce the power-normalized advantage, especially once model mix, concurrency, and serving-runtime choices differ from NVIDIA’s setup.
Limitations & uncertainty
The evidence is a single NVIDIA company technical blog, so the conclusions remain limited to NVIDIA-reported AgentX methods and results rather than independently verified serving performance.