A Comedy of Estimators: On KL Regularization in RL Training of LLMs | alphaXiv
alphaxiv.org · saved by 1 readers
KL regularization in reinforcement learning for language models depends not only on the estimator used, but also on whether it is placed in the reward or directly in the loss. An analysis of K1 and...