What Does Information Theory Say About Designing Agentic Systems? · Hazy Research
TL;DR Many contemporary agentic systems exhibit a clear pattern: larger models interact with the user and orchestrate smaller models that directly interface with the data. In “Deep Research” systems (Google, Anthropic), smaller worker models analyze targeted search results and provide summaries to a larger orchestrator model, which synthesizes a final research report. This separation of control and data plane is also evident in Claude Code and relatives (OpenCode), but it is unclear where scaling actually pays off. This compression-prediction structure isn’t new, an information-theoretic model helps us derive design principles that guide compute spending. Looking at the information actually communicated between the models showed that investing in compressors for relatively cheap mitigates the need for expensive frontier predictors. 📄Paper 💻GitHub Picking the "right" workers is no easy task for Gru. Agentic systems are everywhere now—multiple language model agents (often running in pa