Hidden Bias

Models trained on real-world data can encode real-world bias. Hiding information about protected classes doesn't always fix things — sometimes it can even hurt.

Hidden Bias Explorables Hidden Bias Models trained on real-world data can encode real-world bias. Hiding information about protected classes doesn't always fix things — sometimes it can even hurt. Modeling College GPA Let's pretend we're college admissions officers trying to predict the GPA students will have in college (in these examples we'll use simulated data). One simple approach: predict that students will have the same GPA in college as they did in high school. This is at best a very rough approximation, and it misses a key feature of this data set: students usually have better grades i

related reading

• Subliminal Learning: Language Models Transmit Behavioral Traits via Hidden Signals in Dataalignment.anthropic.com
• Self-Fulfilling Misalignment Data Might Be Poisoning Our AI Modelsturntrout.com
• Statistical Fairness - Turing Commonsalan-turing-institute.github.io
• Models don’t seem to be dishonest in the way humans are — LessWronglesswrong.com
• Auditing language models for hidden objectives \ Anthropicanthropic.com
• AI and prediction of criminal justice risk: revisiting the COMPAS dataset, experiments, and the problem of algorithmic bias and fairnesslink.springer.com
• MAI-Thinking-1: Building a Hill-Climbing Machinemicrosoft.ai
• Can a Model Be Differentially Private and Fair?pair.withgoogle.com
• A Course in Machine Learningciml.info
• Auditing language models for hidden objectives — LessWronglesswrong.com
• How Flawed Data Aggravates Inequality in Credit | Stanford HAIhai.stanford.edu
• Race and Gender Bias As An Example of Unfaithful Chain of Thought in the Wild — LessWronglesswrong.com