flâneur

Value Learning Needs a Low-Dimensional Bottleneck — LessWrong

lesswrong.com · 1,649 words · saved by 1 readers

Comment by beren - Another line of evidence for the 'values are low-dimensional' is all the emergent misalignment work which tends to find that a.) models have a concept of 'general evil' which goes from writing bad code to giving false medical advice and supporting hitler, and b.) this is controlled often a single or a few directions in the residual stream, which implies an extremely small subspace is behind a model's understanding of morality, and hence (presumably?) the general structure of alignment/morality in the dataset. Emergent misalignment is problematic but it also suggests the possibility of 'emergent alignment' where if a model is trained to be good and aligned in many aspect it may also generalise that far to be aligned in many aspects.

x Value Learning Needs a Low-Dimensional Bottleneck — LessWrong Complexity of value Human Values Inverse Reinforcement Learning AI Frontpage 24 Value Learning Needs a Low-Dimensional Bottleneck by Gunnar_Zarncke 23rd Jan 2026 2 min read 7 24 Epistemic status: Confident in the direction, not confident in the numbers. I have spent a few hours looking into this. Suppose human values were internally coherent, high-dimensional, explicit, and decently stable under reflection. Would alignment be easier or harder? My below calculations show that it would be much harder, if not impossible. I'm going to

related reading