Transcript of Sam Altman's interview touching on AI safety — LessWrong
Comment by paulfchristiano - I agree that some forms of robustness research don't have capabilities externalities, but the unreliability of ML systems is a major blocker to many applications. So any robustness work that actually improves the robustness of practical ML systems is going to have "capabilities externalities" in the sense of making ML products more valuable. I disagree even more strongly with "honesty efforts don't have externalities:" AI systems confidently saying false statements is a major roadblock to lots of applications (e.g. any kind of deployment by Google), so this seems huge from a commercial perspective. I agree that interpretability doesn't always have big capabilities externalities, but it's often far from zero. Work that sheds meaningful light on what models are actually doing internally seems particularly likely to have such externalities. In general I think "solve problems that actually exist" is a big part of how the ML community is likely to make progress, and many kinds of safety progress will be addressing problems that people care about today and hence have this kind of capabilities externality. [...] I think 1 unit of safety and 0 units of capabilities is worse than 10 units of safety and 1 unit of capabilities (where a unit is something taking similar labor to uncover), I think it's more like (safety progress) - X * (timelines acceleration) for some tradeoff rate X. Ultimately this seems like it should be a quantitative discussion, and so far from the safety community I'm just not seeing reasonable-looking botecs supporting an emphasis on capabilities externalities. I'm not a very ends-justify-the-means kind of person, but this seems like an application of deontology in a case where most of the good arguments for deontology don't apply. (It also feels like people are using "capabilities" to just mean "anything that makes AI more valuable in the short term," which I think is a really fuzzy definition for which this argument i