Will reward-seekers respond to distant incentives? — LessWrong

Reward-seekers are usually modeled as responding only to local incentives administered by developers. Here I ask: Will AIs or humans be able to influ…

x Will reward-seekers respond to distant incentives? — LessWrong Fitness-seeking AIs Redwood Research AI Frontpage 57 Will reward-seekers respond to distant incentives? by Alex Mallen 16th Feb 2026 AI Alignment Forum 12 min read 4 57 Ω 33 Reward-seekers are usually modeled as responding only to local incentives administered by developers. Here I ask: Will AIs or humans be able to influence their incentives at a distance—e.g., by retroactively reinforcing actions substantially in the future or by committing to run many copies of them in simulated deployments with different incentives? If reward

related reading

• The behavioral selection model for predicting AI motivations — LessWronglesswrong.com
• Fitness-Seekers: Generalizing the Reward-Seeking Threat Model — LessWronglesswrong.com
• Training a Misaligned Reward Seekeralignment.anthropic.com
• Reward Hacking in Reinforcement Learning | Lil'Loglilianweng.github.io
• Fitness-Seekers: Generalizing the Reward-Seeking Threat Modelsubstack.com
• Reward is not the optimization target — LessWronglesswrong.com
• Are AIs more likely to pursue on-episode or beyond-episode reward?blog.redwoodresearch.org
• Reward Is Not Enough — LessWronglesswrong.com
• Fail safe(r) at alignment by channeling reward-hacking into a "spillway" motivationblog.redwoodresearch.org
• The behavioral selection model for predicting AI motivationsblog.redwoodresearch.org
• Reward hacking is becoming more sophisticated and deliberate in frontier LLMs — LessWronglesswrong.com
• Models Don't "Get Reward" — LessWronglesswrong.com