Blog - Aleksa Gordić
aleksagordic.com · 281 words · saved by 3 readers
Long thinking.
Computers can be understood (< back to home) Get notified when I publish a new post. Inside TPU and GPU Clusters: The Anatomy of Collective Communication 2026-07-14 A deep dive into all-gather, reduce-scatter, all-reduce, and all-to-all communication on TPUs and GPUs. Inside the Transformer: The Life of a Token 2026-05-26 A deep dive into a modern dense transformer: YaRN, hybrid attention, soft capping, QK normalization, FLOPs/token, cluster sizing, and more. From Oura Ring to Dataset: Analyzing and Visualizing a Year of Biometric Data 2026-02-16 Visualizing and analyzing sleep,…
saved by
related reading
- How To Scale Your Modeljax-ml.github.io
- How to Think About GPUs | How To Scale Your Modeljax-ml.github.io
- Tiny TPUtinytpu.com
- We reverse-engineered Flash Attention 4modal.com
- TPU Deep Divehenryhmko.github.io
- The Best GPUs for Deep Learning in 2023 — An In-depth Analysistimdettmers.com
- Inside NVIDIA GPUs: Anatomy of high performance matmul kernels - Aleksa Gordićaleksagordic.com
- How to Parallelize a Transformer for Training — an explorable explanationezyang.github.io
- Transformer Inference Arithmetic | kipply's blogkipp.ly
- Modern GPU Programming For MLSys — Modern GPU Programming For MLSysmlc.ai
- How to Think About TPUs | How To Scale Your Modeljax-ml.github.io
- Overleaf Examplearxiv.org