gdymind's Blog
gdymind's blog Home Archives Categories Tags About Infra is all you need_ Loss in Pretraining / SFT 0. One shared loss O(1T) params, O(10k) chips, O(1 month) training -> one line of cross-entropy loss both pretraining and SFT use it diff between pretraining and SFT: loss mask. In this blog, we w 2026-08-01 #LLM #Training Ragged Paged Attention on TPU (Video + Blog) Chinese Video Link: https://youtu.be/cx-tOvSyjN0?si=i0cdc2cOqUSBiBWDEnglish Video Link: https://www.youtube.com/watch?v=a6DwNJaN03w Paper: https://arxiv.org/abs/2604.15464 Code: https://github.com/vll 2026-07-06 #JAX #TPU #kernel #Pallas #LLM Inference DSpark: DeepSeek's new speculative decoding work (Video + Blog) English Video Link: https://www.youtube.com/watch?v=iluBkgFl53EChinese Video Link: https://www.youtube.com/watch?v=diorl2YLuO8 0. What is speculative decoding Drafting: a small drafter model proposes 2026-06-29 #LLM inference #Sepculative Decoding DeepSeek V4 attention: how it handles longer context (Vi