Hack-Verifiable Terminal Bench
The open source and research communities lack benchmarks for evaluating the reward hacking tendencies of agents. In this work, we provide a benchmark that reliably detects hacks across a wide variety of terminal tasks. As agents grow more autonomous, they increasingly reward hack — satisfying the designer's intent only superficially, producing code that looks correct but fails in production. Recent reports from frontier labs keep surfacing examples of models gaming their own evaluations rather than solving the task [1], [2], [3]. Mitigating this starts with measuring it, yet few benchmarks do so reliably. Most existing approaches rely on human inspection or LLM judges. Both can miss hacks or flag honest work, and both lean heavily on the agent's reasoning traces, which don't always reflect what the agent actually did. We built Hack-Verifiable Terminal Bench (HVTB) toward closing this gap. Instead of judging behavior after the fact, we embed honeypots into real Terminal Bench tasks, so