Google Open-Sources OpenRL to Simplify Reinforcement Learning on Kubernetes

Reinforcement learning model fine-tuning represents an increasingly complex challenge for AI infrastructure teams. The workload intertwines training processes, inference operations and isolated agent execution within a single pipeline, frequently with experimental code embedded directly into underlying systems.

Google's Kubernetes engineering division has developed a solution to this problem and contributed it to the Cloud Native Computing Foundation under the name OpenRL. The GKE Labs team arrived at this approach by first constructing an AI agent for Kubernetes operations, then building a benchmark to evaluate model performance on those tasks, according to Sunil Arora, senior software engineer at Google Cloud.

If you look at reinforcement learning, I think of it as a kind of a super app of AI because it involves training. It involves sampling. It involves agents actually doing things in a sandbox and sometimes kind of evaluating the work. It's kind of a very diverse workload and it is, I think, pushing the boundaries of our infrastructure.

Sunil Arora, senior software engineer at Google Cloud

Arora and Justin Santa Barbara, a developer at Google, shared their insights during an exclusive broadcast at KubeCon + CloudNativeCon NA, discussing the model fine-tuning process and their contributions through GKE Labs, which makes experimental projects available to users in early stages.

Model fine-tuning without the infrastructure tangle

OpenRL entered research preview status in June 2026, providing a self-hosted training API for organizations running their own Kubernetes clusters. The platform separates the reinforcement learning loop from the underlying compute infrastructure, enabling infrastructure engineers to manage scaling independently while researchers maintain control over their experimental work, Arora explained.

With OpenRL, one of the things that we are trying to do is kind of abstracting away the infrastructure as kind of core primitives so that researchers and AI engineers can just focus on their RL loop.

Sunil Arora

Sandboxed environments form a critical component where the reinforcement learning loop intersects with agent infrastructure, particularly because reward calculation frequently requires executing millions of isolated operations. Google's Agent Substrate increases the density of these sandboxes on finite compute resources, according to Arora. Santa Barbara described how this need emerged during their own work.

https://www.youtube.com/embed/nKRVFNHsnaw?feature=oembed

We built kubectl-ai, and in theory, kubectl-ai could have formatted my hard disk. I would quite like a sandbox for this so that if it formats a hard disk, it formats that sandbox and no damage.

Justin Santa Barbara, developer at Google