Nebius cut rollout collection time by 66.9% — here’s what changed

The time required to collect rollout data in AI agent training saw a reduction of 66.9% at Nebius, according to the announcement.

In the realm of AI agent training, powerful GPUs alone do not guarantee speed; much of the time can be spent simply waiting. Agents must execute tasks and yield results before the underlying model can learn from them. Slow attempts can stall subsequent training updates, making the systems responsible for running those tasks a critical factor in training cost and speed.

Di Jin, VP of AI Training at Nebius, noted that teams accustomed to pretraining or supervised fine-tuning might underestimate the demands placed on these systems. He stated, “If you staff and design it as a training project, inference becomes the bottleneck nobody owns.” Rollout generation frequently consumes the majority of a training step’s wall-clock duration, putting batching, scheduling, and cache management at the core of how quickly the model can learn.

Jin further suggested that engineers should track the time to achieve a trainable batch, not just throughput. Average tokens per second can obscure the long tail of execution times; in a synchronous batch, the step cannot conclude until the final trajectory is finished, and agent trajectories vary widely in length.

When success on a difficult task is infrequent, reusing an already successful trajectory, even if slightly outdated, holds value. However, staleness becomes detrimental when the generating policy has diverged too far from the policy currently under training. Jin asserted, “Staleness isn’t the enemy. Unmeasured staleness is.” Instead of trying to eliminate old data, organizations should quantify and manage its staleness, perhaps by applying importance-sampling corrections or tracking divergence.

Jin observed that in his experience, environments and data constitute the bulk of the effort in successful RL post-training, with algorithm design being a minor component, while infrastructure enables swift iteration. He emphasized, “Infrastructure determines how fast you can iterate. But what the model actually learns is decided by the tasks, environments, and rewards you give it.” For agents, this means concentrating on the environments that generate the necessary training experience.

Failures often surface at the interfaces between components like the sandbox, verifier, and trainer, even if each piece functions correctly in isolation. Engineers must log whether an episode concluded due to model success, model failure, infrastructure failure, or an unknown outcome. Failures stemming from the model’s own actions should still register as model failures, ensuring timeouts do not become an evasion tactic.

For GRPO-style training, where multiple task attempts are scored against each other, scheduling should align with group completion, not launch order. Nebius found that adjustments to routing, batch assembly, and sandbox execution reduced fixed-weight collection time for a 256-trajectory batch by 66.9%.

A key infrastructure objective for agentic AI rollouts is minimizing the time expensive GPUs spend waiting on non-GPU dependencies like browsers or code execution environments. Jin advised scaling CPU and sandbox capacity independently and generously, noting, “Your most expensive GPU is often waiting on your cheapest container.”

When numerical differences arise between the rollout engine and the trainer—due to quantization or batching—systematic bias can enter the training updates. Jin recommends aligning the numerics first, which might involve matching the quantization scheme used in the trainer’s forward pass to the rollout engine.

To maintain policy freshness, rollout workers require a dependable mechanism to receive updated weights as the policy evolves. Jin outlined three primary methods: resharding weights on shared GPUs, transferring weights to distinct inference workers, or publishing checkpoints for hot-loading.

Teams must measure end-to-end publication latency separately from the pause required to apply weights. Furthermore, if a trajectory spans multiple policy versions, recording which version generated its tokens allows the training system to handle that mixture deliberately. Jin suggested that smaller teams begin simply, adding asynchronous complexity only when measured bottlenecks warrant it.