Brainformers: Trading Simplicity for Efficiency
arxiv.org · 5,735 words · saved by 1 readers
N/A
Brainformers: Trading Simplicity for Efficiency Yanqi Zhou 1 Nan Du 1 Yanping Huang 1 Daiyi Peng 1 Chang Lan 1 Da Huang 1 Siamak Shakeri 1 David So 1 Andrew Dai 1 Yifeng Lu 1 Zhifeng Chen 1 Quoc Le 1 Claire Cui 1 James Laudon 1 Jeff Dean 1 Abstract Scaling…
related reading
- A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
- How To Scale Your Modeljax-ml.github.io
- frontier model training methodologies | Alex Wa's Blogdjdumpling.github.io
- Transformer Math 101 | EleutherAI Blogblog.eleuther.ai
- Mamba: The Easy Wayjackcook.com
- Transformers Inference Optimization Toolset | AstraBlogastralord.github.io
- On the Tradeoffs of SSMs and Transformers | Goomba Labgoombalab.github.io
- [2101.03961] Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsityarxiv.org
- All the Transformer Math You Need to Know | How To Scale Your Modeljax-ml.github.io
- The Big LLM Architecture Comparisonmagazine.sebastianraschka.com
- The Annotated Transformernlp.seas.harvard.edu
- Linear Transformers Are Faster After All – Manifest AImanifestai.com