Introducing BART | TensorGoose
sshleifer.github.io · 2,479 words · saved by 1 readers
Episode 1 – a mysterious new Seq2Seq model with state of the art summarization performance visits a popular open source library
Overview Background: Seq2Seq Pretraining Bert vs. GPT2 Encoder-Decoder Pretraining: Fill In the Span Summarization Demo: BartForConditionalGeneration Conclusion Overview For the past few weeks, I worked on integrating BART into transformers . This post covers the high-level differences between BART and its predecessors and how to use the new BartForConditionalGeneration to summarize documents. Leave a comment below if you have any questions! Background: Seq2Seq Pretraining In October 2019, teams from Google and Facebook published new transformer papers: T5 and BART . Both papers achieved bette
related reading
- Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
- Transformers from Scratche2eml.school
- Getting Caught Up to Modern LLM Research | Samarth Goeldev.samarthgoel.com
- GitHub - brexhq/prompt-engineering: Tips and tricks for working with Large Language Models like OpenAI's GPT-4.github.com
- Parsed | Custom, interpretable AI systems that continuously learnparsed.com
- Seq2seq and Attentionlena-voita.github.io
- Generalized Language Models | Lil'Loglilianweng.github.io
- The Bitter Lesson is coming for Tokenization – ⛰️ lucalplucalp.dev
- The Annotated Transformernlp.seas.harvard.edu
- Gwern visits BAIR – Yuxi on the Wiredyuxi.ml
- Tinkerthinkingmachines.ai
- Yi Tayyitay.net