Tencent Hy
• Abstract Larger batches expose more parallelism by processing more sequences per update, but under a fixed sample budget, they also leave fewer opportunities to update the model. When does this trade-off reduce wall-clock time-to-target in reinforcement learning? For large language models, answering this question requires accounting for the online generation of training trajectories and the computational asymmetry between generation and training. We develop a framework that connects samples-to-target with end-to-end throughput: a larger batch accelerates training only when its throughput gain exceeds its samples-to-target penalty. This framework also suggests a simple practical strategy—first establish approximate batch-size invariance, then optimize throughput within that regime. Experiments with GRPO and PPO show that learning-rate retuning approximately preserves sample-indexed learning trajectories over a bounded batch range. On fixed hardware, larger batches improve generation t