The Best of LessWrong — LessWrong
How does it work to optimize for realistic goals in physical environments of which you yourself are a part? E.g. humans and robots in the real world, and not humans and AIs playing video games in virtual worlds where the player not part of the environment. The authors claim we don't actually have a good theoretical understanding of this and explore four specific ways that we don't understand this process. Paul thinks common stories about what AI catastrophe might look like are flawed. Here, he paints a picture of two failure scenarios that seem more likely, summarized as: "ML helps us get what we measure (creating a slow rolling catastrophe)", and "ML gives rise to greedy patterns that try to expand their influence." The original draft of Ayeja's report on biological anchors for AI timelines. The report includes quantitative models and forecasts, though the specific numbers were still in flux at the time. Ajeya cautions against wide sharing of specific conclusions, as they don't yet r