Xiang Fu

xiangfu.co · 514 words · saved by 1 readers

Discovery is the skill of creating and detecting deviations between expectation and observation, to form hypotheses, test them, and refine ideas in light of both past and new evidence. A deviation matters only when a precise framework of expectation reveals the gap and generates a general and testable hypothesis. Today’s AIs are powerful problem solvers, optimized to excel at tasks with well-defined solutions, as demonstrated in competitions like the IMO and ICPC. But discovery demands the opposite orientation: it requires seeking out anomalies, which by nature live in the long tail. Current training regimes instead push models to suppress anomalies rather than create them. Pretraining/SFT aligns models to the observed data distribution, and RL tends to reward mode-seeking within tasks where correctness is well-defined. The result is that an LLM is reinforced for fitting expectations, not for exposing where those expectations break. RL with an objective to discover new things requires

An Age of AI Enlightenment Sep 30, 2025 Discovery is the skill of creating and detecting deviations between expectation and observation, to form hypotheses, test them, and refine ideas in light of both past and new evidence. A deviation matters only when a precise framework of expectation reveals the gap and generates a general and testable hypothesis. Creating Deviation Today’s AIs are powerful problem solvers, optimized to excel at tasks with well-defined solutions, as demonstrated in competitions like the IMO and ICPC. But discovery demands the opposite orientation: it requires seeking out

Xiang Fu

related reading