AI Business

Liqid's Dynamic Memory Pooling Reshapes AI Inference Economics

As AI deployments become unpredictable, companies are moving away from fixed GPU-to-memory ratios in servers toward flexible resource pools that adapt to whatever model needs to run next.

·2 min read
AI’s next build-out: Inference racks that resize themselves for every new model
AI’s next build-out: Inference racks that resize themselves for every new model

The race to maximize output from each graphics processing unit has made memory pooling a critical lever in AI inference economics. Sumit Puri, co-founder and chief technical officer of Liqid Inc., argues that the industry's reliance on servers with fixed GPU-to-memory ratios no longer makes sense when model deployments remain uncertain months in advance.

Puri outlined Liqid's approach during an exclusive conversation at SC26 with theCUBE Research's John Furrier and Gemma Allen. "Our core fundamental technology is around pooling and sharing the most critical resources in the data center, which are now GPU and memory," Puri said. "Rather than statically deploying these things inside of a server, our vision is simple. Let's put pools of resources in the rack, GPUs and memory, and dynamically assign them to any server that needs it based upon the model or the workload that we are looking to deploy."

Memory pooling meets the tokenomics test

Liqid evaluates success through tokenomics metrics: tokens per second, tokens per watt, and most importantly, tokens per dollar. Memory represents one of the most constrained components in AI infrastructure buildouts, and Puri contends that allocating only the memory each application requires can help alleviate supply chain pressures.

"Supply chain is a massive constraint for all of us — memory being a giantly massive constrained thing," Puri explained. "If we can do things like pool memory, share memory, optimally deliver memory to the server in the right amount of memory as required by the application, we can directly address some of those supply chain issues."

Liqid is expanding its go-to-market approach for memory pooling. The company unveiled Liqid as a Service at SC26, a fully managed single-tenant option, while also moving toward preconfigured appliances. Puri believes most enterprises will run models rather than develop them from scratch, meaning they will need dedicated capacity.

https://www.youtube.com/embed/ERXkBsWsjF0?start=2&feature=oembed

"The world went to unlimited consumption of minutes, and that's where we believe tokens are headed," Puri said. "Everybody's going to own their own token machine so they can consume these tokens at will."