How does GPT-3 spend its 175B parameters? - LessWrong

[Target audience: Me from a week ago, and people who have some understanding of ML but want to understand transformers better on a technical level.] …

x How does GPT-3 spend its 175B parameters? — LessWrong Interpretability (ML & AI) GPT AI World Modeling Frontpage 41 How does GPT-3 spend its 175B parameters? by Robert_AIZI 13th Jan 2023 Linkpost for aizi.substack.com 7 min read 14 41 [Target audience: Me from a week ago, and people who have some understanding of ML but want to understand transformers better on a technical level.] Free advice for people learning new skills: ask yourself random questions. In answering them, you’ll strengthen your understanding and find out what you really understand and what’s actually useful. And some day, i

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• ali (@waterloo_intern) on Xx.com
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
• Transformer Taxonomy (the last lit review)kipp.ly
• A Conceptual Guide to Transformers: Part Ibenlevinstein.substack.com
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• interpreting GPT: the logit lens — LessWronglesswrong.com
• Transformers from Scratche2eml.school
• GPT in 60 Lines of NumPy | Jay Modyjaykmody.com
• The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
• All the Transformer Math You Need to Know | How To Scale Your Modeljax-ml.github.io