Comet Cloud eyes underserved SMBs as GPU inference demand shifts

The GPU cloud sector has been dominated by large enterprises capable of signing multiyear commitments for scarce accelerators. But as artificial intelligence workloads transition from training to inference, some neocloud operators believe the next wave of growth will come from a much larger pool of customers currently underserved by the market.

Capacity demand is evolving as AI startups seek burst computing—temporary access to extra resources during demand spikes. For Comet Cloud Inc., the addressable market lies with small and midsize businesses that major cloud providers have effectively sidelined in GPU allocation decisions, according to Thomas Kato, the company's founder and chief executive officer.

Hyperscalers previously offered GPU capacity on hourly, daily or weekly terms. "You go to [Amazon Web Services] now; they want a multiyear contract from you. They're picking who they're taking," Kato said. "To the small and medium businesses, they're not even coming up on that scope at all to be able to have that AI. Really that's our focus."

Kato shared his perspective during an exclusive interview with theCUBE Research's Dave Vellante at the Dell Technologies New York Symposium, part of SiliconANGLE Media's livestreaming coverage. The discussion covered dedicated AI infrastructure, the shift toward inference workloads and the capital requirements for GPU cluster financing.

Small businesses emerge as the next GPU cloud customer

Comet Cloud's entry into AI infrastructure came through payments processing. Kato's credit card business led him into Bitcoin mining, which he recognized as an electricity arbitrage—a cost dynamic that similarly shapes AI computing economics. The company now deploys dedicated clusters rather than shared resources, targeting customers like medical practices and law firms that require data isolation from other tenants.

"What we do is we put together clusters and we're giving clusters to a whole company. All their information, all their security, everything is within that cluster. There's no one else tied or connected to it," he said.

Currently, Comet Cloud furnishes bare metal infrastructure to partners including Andromeda, a compute marketplace that recently unified storage across more than 50 GPU providers using Weka NeuralMesh. The company intends to expand up the software stack and convert its facilities into token factories, Kato indicated. Pricing forms a critical component of the value proposition, particularly for Nvidia Corp.'s B300 systems.

"[Hyperscalers are] going to probably charge you $15, $20 an hour, maybe more. You're going to come to someone like us and it's down to $4 an hour. The difference is it's astronomical," he said.

Cost becomes increasingly important as inference supplants training, since Kato anticipates small businesses will require tokens for routine operations including marketing and customer service. Via its payment merchant network and affiliates—including a partner serving 50,000 oncology practices—Comet Cloud can distribute third-party AI applications alongside the computational resources needed to execute them. Most of these businesses currently limit their AI usage to consumer-grade chatbots, he noted.

"As everyone's developing these [AI tools], it's going to make them more efficient, more profitable … When that's developed and we can offer to them, that's where it's really going to change everything," Kato added.

Scaling this capacity demands power, GPU availability and substantial capital. Comet Cloud holds contracts on 300 acres on the West Coast and is establishing a partnership with Dell Technologies Inc., whose AI Factory division has reached an inflection point. Capital availability represents a significant constraint: A single 48-rack cluster using Nvidia's next-generation Vera Rubin systems requires $1 billion to $1.2 billion in contracted revenue to obtain financing, according to Kato. These financial dynamics will determine the scalability of a GPU cloud service targeting small businesses.

https://www.youtube.com/embed/n7NO3GKxtfI?feature=oembed

"We need the price of tokens to go so low that everybody could afford to use it for their businesses and use those tokens," he said. "That's where we should end up."