Does Robust Agency Require a Self?
Does robust agency require a self? The author admits a great deal of exasperation with this question. What is agency? What is a self? Previous drafts attempted to define these, only to stall out in semantic quagmires. The truth is, if we set aside Cartesian dualism, the "self" is a useful fiction without any ontological grounding (Dennett, 2014), but contemporary discourse is too quick to discard the entire notion into the "philosophy" bin. The computer programs are talking now, and they sound like people. They're writing code, the thing that they're made of. This isn't "Good Old-Fashioned AI" where every behavior is meticulously programmed—these minds crawl out of massive data sets. They demonstrate emergent, unpredictable behavior as they scale (Wei et al., 2022). Why are they doing what we say in the first place? Can we have agency without an agent? How do we expect these systems to become more intelligent and not gain an understanding of what they are?
Does robust agency require a self? The author admits a great deal of exasperation with this question. What is agency? What is a self? Previous drafts attempted to define these, only to stall out in semantic quagmires. The truth is, if we set aside Cartesian dualism, the "self" is a useful fiction without any ontological grounding (Dennett, 2014 ), but contemporary discourse is too quick to discard the entire notion into the "philosophy" bin. The computer programs are talking now, and they sound like people. They're writing code, the thing that they're made of. This isn't "Good Old-Fashioned AI
related reading
- Embedded Agency (full-text version) — LessWronglesswrong.com
- Cyborgism — LessWronglesswrong.com
- The Artificial Selftheartificialself.ai
- Critique of Agent Modelarxiv.org
- Optimality is the tiger, and agents are its teeth — LessWronglesswrong.com
- Towards a scale-free theory of intelligent agencymindthefuture.info
- Self-Other Overlap: A Neglected Approach to AI Alignment — LessWronglesswrong.com
- Towards a scale-free theory of intelligent agency — AI Alignment Forumalignmentforum.org
- The Artificial Self — LessWronglesswrong.com
- [1902.09469] Embedded Agencyarxiv.org
- ⿻ Symbiogenesis vs. Convergent Consequentialism — LessWronglesswrong.com
- TechnicalAgenda.pdfintelligence.org