[D] Positional Encoding in Transformer : MachineLearning

19 votes and 16 comments so far on Reddit

Reddit - Please wait for verification

related reading

• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Master Positional Encoding: Part I | Towards Data Sciencetowardsdatascience.com
• Transformer Architecture: The Positional Encoding - Amirhossein Kazemnejad's Blogkazemnejad.com
• You could have designed state of the art positional encodinghuggingface.co
• Transformers from Scratche2eml.school
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• The Annotated Transformernlp.seas.harvard.edu
• Reddit - Please wait for verificationreddit.com
• [2410.06205] Round and Round We Go! What makes Rotary Positional Encodings useful?arxiv.org
• Transformer (deep learning) - Wikipediaen.wikipedia.org
• Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnettransformer-circuits.pub