Prefix-Tuning: Optimizing Continuous Prompts for Generation

Fine-tuning is the de facto way to leverage large pretrained language models to perform downstream tasks. However, it modifies all the language model parameters and therefore necessitates storing a full copy for each task. In this paper, we propose prefix-tuning, a lightweight alternative to fine-tuning for natural language generation tasks, which keeps language model parameters frozen, but optimizes a small continuous task-specific vector (called the prefix). Prefix-tuning draws inspiration from prompting, allowing subsequent tokens to attend to this prefix as if it were "virtual tokens". We apply prefix-tuning to GPT-2 for table-to-text generation and to BART for summarization. We find that by learning only 0.1\% of the parameters, prefix-tuning obtains comparable performance in the full data setting, outperforms fine-tuning in low-data settings, and extrapolates better to examples with topics unseen during training.

Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang Stanford University Stanford University xlisali@stanford.edu pliang@cs.stanford.edu Abstract Fine-tuning is the de facto way to leverage…

related reading

• Recent Advances in Language Model Fine-tuningruder.io
• [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• Patterns for Building LLM-based Systems & Productseugeneyan.com
• Prompt Tuning and Prefix Tuningericwiener.github.io
• GenAI Handbookgenai-handbook.github.io
• Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Togetherarxiv.org
• Generalized Language Models | Lil'Loglilianweng.github.io
• The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities (Version 1.0)arxiv.org
• Large Language Models Reading List | Sebastian Raschka, PhDsebastianraschka.com
• Parameter-Efficient Fine-Tuning using 🤗 PEFThuggingface.co
• Conditioning, Prompts, and Fine-Tuning — LessWronglesswrong.com