What the Models Learned in One Layer Deeper | Core Automation
coreauto.com · 882 words · saved by 1 readers
What worked across 15,602 submissions to the One Layer Deeper competition.
The challenge Reasoning models can spend more compute by generating more tokens. Could they learn to use more steps inside the model instead? Our launch post asked whether a model could “productively use more computational depth at test time than it encountered during training.” The public problem is repeated modular squaring: Repeated modular squaring z = x % N for _ in range(T): z = z * z % N Each squaring depends on the previous result. Increasing T lengthens that chain without requiring longer inputs or answers. The aim was to learn the computation from examples, then run it for…
saved by
related reading
- Quantifying the Necessity of Chain of Thought through Opaque Serial Deptharxiv.org
- (Some of) The Models, They Just Don't Want to Learn | Tildeblog.tilderesearch.com
- As Rocks May Think | Eric Jangevjang.com
- The Little Book of Deep Learningfleuret.org
- AlgZoo: uninterpreted models with fewer than 1,500 parameters — LessWronglesswrong.com
- DeepSeek-R1arxiv.org
- the-illusion-of-thinking.pdfml-site.cdn-apple.com
- Learning to reason with LLMs | OpenAIopenai.com
- Composer2.pdfcursor.com
- Clouded Judgement 9.4.26 - Recurrent Depthcloudedjudgement.substack.com
- o1 and Reasoning | AndoLogsblog.ando.ai
- Do Machine Learning Models Memorize or Generalize?pair.withgoogle.com