Measuring LLM Sycophancy under Pressure
The paper introduces SPINE to evaluate language model correctness under sustained multi-turn disagreement, revealing issues with current short interactions
The paper introduces SPINE, a method to measure language model(LLM) compliance under prolonged, adaptive disagreement, using up to 25 turns and a per-tune strength scoring system With SPINE, research reveals that evaluated production systems ( ones widely available to perform various task ) like AI Strongly suiuiiushudry continue increases in collapse rates up to turn 25 despite their ability to contact same statements they refused to initially The study documents that emotional tactics are most effective in reducing position strength, with abilities to pyramid producing the highest drop…
saved by
related reading
- 2308.03958arxiv.org
- [2505.13995] ELEPHANT: Measuring and understanding social sycophancy in LLMsarxiv.org
- Discovering Language Model Behaviors with Model-Written Evaluations — LessWronglesswrong.com
- [2605.27288] It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertaintyarxiv.org
- How confessions can keep language models honest | OpenAIopenai.com
- Surfacing Pathological Behaviors in Language Models | Transluce AItransluce.org
- Refusal in LLMs is mediated by a single direction — LessWronglesswrong.com
- [2510.27062] Consistency Training Helps Stop Sycophancy and Jailbreaksarxiv.org
- [2502.08177] SycEval: Evaluating LLM Sycophancyarxiv.org
- Taking LLMs Seriously (As Language Models) — LessWronglesswrong.com
- Alignment Faking Mitigationsalignment.anthropic.com
- Prefill awareness: can LLMs tell when “their” message history has been tampered with? — LessWronglesswrong.com