Nvidia and CoreWeave Address CPU Constraints in Agentic AI Systems

Building infrastructure for agentic AI requires handling not just model reasoning but also the computational work agents perform between decisions. Graphics processing units drive the reasoning side, while central processing units manage execution tasks. This division of labor underpins Nvidia's strategy for deploying its Vera CPU alongside CoreWeave's platform, according to Hannah Coutand, director of product marketing for Vera CPU at Nvidia.

Coutand explained the motivation behind the Vera CPU initiative: "The reason why Vera is so important is because we recognize that extreme co-design means that we look across the AI factory and we want to make sure we solve for any inefficient bottlenecks. We recognized the CPU was becoming one of them. We didn't set out to say, 'Let's go and build CPUs.' We set out to solve this bottleneck."

Coutand and Harsh Banwait, senior director of product at CoreWeave, discussed these infrastructure challenges during the Fully Connected event, speaking with theCUBE Research's Dave Vellante and John Furrier. Their conversation covered CPU specifications, sandbox performance and security measures.

CPU Demands Intensify as AI Agents Execute Tasks

The convergence of training and inference operations means infrastructure must support both model computation and the execution environments where agents operate. Tool invocations, API calls and database queries all generate CPU-intensive work, Coutand noted.

"I think the volume certainly plays a role," she said. "That's why having a CPU that handles those types of calls extremely well, with fast, beefy cores, high memory bandwidth and low latency, it handles both operating in this new world … and serves as a great foundation for simply agentic use cases [and] reinforcement learning."

CoreWeave plans to offer Vera CPU capacity as a standalone service within its broader infrastructure portfolio. The company's Sandboxes product creates isolated execution spaces for agents during reinforcement learning and inference operations, Banwait explained.

Testing Vera CPUs with Sandboxes yielded significant performance gains. "Quite recently, we also tested that with Vera CPUs, and we were proud to share that we saw about a 3x improvement in performance in terms of Sandbox startup times," Banwait said. "For us, it's a very important combination of getting the performance that we need from the silicon and getting our customers the experience that they expect from a product."

Security and Performance Integration Across Infrastructure

Security considerations shape how agentic AI infrastructure must be designed. Nvidia's Open Agent Safety Platform integrates OpenShell runtime controls with Nvidia Sentry, which runs on BlueField-4 data processing units to provide independent monitoring and enforcement capabilities.

The architecture consolidates these components into a single form factor. "They're all in a single Vera Rubin tray, which includes Vera CPU as well as the DPU; that's all included in the same hardware infrastructure," Coutand said. "So, Open Agent Safety Platform, the secure runtime, which is OpenShell, runs on the Vera CPU in that tray, and DOCA Sentry runs on the DPU part."

Deploying this infrastructure demands careful attention to power consumption, thermal management and operational automation. Networking, processors and storage systems all require coordinated advancement, Banwait emphasized.

https://www.youtube.com/embed/qTBDZljPmqQ?feature=oembed

"All of that needs to be able to keep up," he said. "We're going to continue to focus on wherever the bottleneck is so that when the entire system is kind of advancing, it's doing that in one cohesive way. Otherwise, the weakest link in the chain kind of holds it all back."