Enterprise AI Workloads Drive Shift Away From Centralized GPU Clusters

Organizations developing AI-native applications are learning that having access to GPUs is just the starting point. The transition from cloud-native to AI-native computing is reframing the challenge: instead of asking which provider controls the most accelerators, enterprises now want to know who can deliver distributed, secure workloads across multiple regions.

Early neocloud providers built massive GPU clusters to serve a small number of hyperscale customers, but this approach leaves enterprise developers short. The Constant Company LLC, operating as Vultr, is pursuing a different strategy by combining AMD processors with open-source software to address what it sees as a gap in the market. Kevin Cochrane, Vultr's chief marketing officer, outlined the company's vision during an interview at KubeCon + CloudNativeCon NA.

In the early market … it was all about just standing up large GPU clusters wherever you could get land, power and shell, and then just selling it on through to a hyperscaler. And somehow in that mix, people forgot that you still need cloud, right? You still gotta run this thing.

Kevin Cochrane, chief marketing officer of Vultr

Why AI-native applications need a distributed cloud

Each major computing shift requires rethinking infrastructure from the data center level all the way up to applications, according to Cochrane. The initial wave of AI infrastructure was designed around the needs of a small set of the largest GPU consumers. Enterprise customers, by contrast, have distinct requirements that demand a return to longer-standing computing principles.

We took a little bit of a step backwards because in that early iteration of the new architecture, when we over-indexed on a limited number of buyers, we went back more to centralized systems. And we have to remember that the long arc of compute is towards decentralized distributed systems driven by open standards and driven by open source.

Kevin Cochrane

The shift to distributed systems also transforms how inference operates at scale. When many users submit similar queries, token reuse becomes valuable, and Vultr is working with partners including Vast Data Inc. to share KV cache across a global namespace. Equally important is the networking layer: enterprises require private connections between their data centers that remain isolated from other tenants sharing public cloud infrastructure.

In the neocloud moment, I think some of the complexity of what's required to operate workloads at global scale for enterprises has been forgotten or just assumed it wouldn't matter. But … it's the biggest barrier to getting enterprises to actually leverage GPUs from a new cloud because the networking connectivity, it just doesn't work.

Kevin Cochrane

Vultr is investing in the open-source ecosystem that underpins AI infrastructure development. The company joined the Cloud Native Computing Foundation as a platinum member and introduced its Open Source Credits Program at KubeCon, which provides computing resources to maintainers of emerging CNCF projects.

https://www.youtube.com/embed/4EXdTKIN2XE?feature=oembed

All of those projects need compute, right? The maintainers aren't going to suddenly open up their own personal checkbooks. How can you build the next great CNCF project if you don't have compute?

Kevin Cochrane