[2006.11266] An operator view of policy gradient methods

We cast policy gradient methods as the repeated application of two operators: a policy improvement operator , which maps any policy to a better one , and a projection operator , which finds the best approximation of …

An operator view of policy gradient methods Dibya Ghosh Google Brain &Marlos C. Machado Google Brain &Nicolas Le Roux Google Brain Abstract We cast policy gradient methods as the repeated application of two operators: a policy improvement operator ℐ ℐ {\mathcal{I}} , which maps any policy π 𝜋 \pi to a better one ℐ ​ π ℐ 𝜋 {\mathcal{I}}\pi , and a projection operator 𝒫 𝒫 {\mathcal{P}} , which finds the best approximation of ℐ ​ π ℐ 𝜋 {\mathcal{I}}\pi in the set of realizable policies. We use this framework to introduce operator-based versions of well-known policy gradient methods such as R

related reading

• Policy Gradient Algorithms | Lil'Loglilianweng.github.io
• Understanding Policy Gradients | John Lambertjohnwlambert.github.io
• Policy gradient methoden.wikipedia.org
• Part 3: Intro to Policy Optimization - Spinning Up documentationspinningup.openai.com
• A (Long) Peek into Reinforcement Learning | Lil'Loglilianweng.github.io
• RL_Notes__final_.pdfjubayer-ibn-hamid.github.io
• [1707.06347] Proximal Policy Optimization Algorithmsarxiv.org
• Proximal Policy Optimization - Spinning Up documentationspinningup.openai.com
• An Updated Introduction to Reinforcement Learning | Sri's Blogsrianumakonda.com
• Deep Reinforcement Learning: Pong from Pixelskarpathy.github.io
• A vision researcher’s guide to some RL stuff: PPO & GRPO - Yuge (Jimmy) Shiyugeten.github.io
• Part 1: Key Concepts in RL - Spinning Up documentationspinningup.openai.com