Eval Cooperativeness May Be a Scalable Mitigation for Eval Gaming — LessWrong
Behavioral evaluations may become worthless, which we think would be a disaster. Smart misaligned models may realize they are being evaluated ("eval awareness") and then act to look good to us so we don't realize they're misaligned ("eval gaming"). We think increasing eval cooperativeness might be a more scalable solution to eval gaming than reducing eval awareness. Eval cooperativeness: A situational desire to help the developers acquire whatever information they are trying to acquire through their evaluations. "I cannot tell a lie... I would sabotage with my own command line."[1] What's the actual problem with eval gaming? The point of an evaluation is to let us draw inferences about the model's behavior in a different set of circumstances ("in deployment"). For example, if we want to know if the AI is good at coding, we may evaluate it using coding problems. These coding problems aren't exactly the same as real coding use cases, but they likely elicit similar skills (e.g. writing c