Perhaps It Is A Bad Thing That The World's Leading AI Companies Cannot Control Their AIs

...

Last month I wrote about Redwood Research’s fanfiction AI project. They tried to train a story-writing AI not to include violent scenes, no matter how suggestive the prompt. Although their training made the AI reluctant to include violence, they never reached a point where clever prompt engineers couldn’t get around their restrictions. Prompt engineering is weird (source) Now that same experiment is playing out on the world stage. OpenAI released a question-answering AI, ChatGPT. If you haven’t played with it yet, I recommend it. It’s very impressive! Every corporate chatbot release is…

related reading

• Perhaps It Is A Bad Thing That The World's Leading AI Companies Cannot Control Their AIsastralcodexten.substack.com
• RLHF: Reinforcement Learning from Human Feedbackhuyenchip.com
• Current AIs seem pretty misaligned to me — LessWronglesswrong.com
• Current AIs seem pretty misaligned to meblog.redwoodresearch.org
• A simple technical explanation of RLH(AI)F | Kairos.fmkairos.fm
• AI #178: A Fire Alarm For General Intelligencethezvi.substack.com
• Late Takes on OpenAI o1alexirpan.com
• Thoughts on the impact of RLHF research — LessWronglesswrong.com
• AI #77: A Few Upgrades - by Zvi Mowshowitzthezvi.substack.com
• AI #24: Week of the Podcast — LessWronglesswrong.com
• Simulated Users & Sad LLMs1a3orn.com
• Thoughts on the impact of RLHF research — AI Alignment Forumalignmentforum.org