Probe-Based Data Attribution: Surfacing and Mitigating Undesirable Behaviors in LLM Post-Training - Goodfire
goodfire.com · saved by 1 readers
Post-training can introduce undesired side effects that are difficult to detect and even harder to trace to specific training datapoints. We show that a probe-based method can surface concerning behaviors that emerge during LLM post-training, and that probes can identify the datapoints responsible for a specific harmful behavior. Filtering out those datapoints and retraining significantly reduces the behavior.