{"url":"/method/sft","slug":"sft","name":"SFT","full_name":"Shrink and Fine-Tune","full_name_withheld":false,"description_markdown":"**Shrink and Fine-Tune**, or **SFT**, is a type of distillation that avoids explicit distillation by copying parameters to a student student model and then fine-tuning. Specifically it extracts a student model from the maximally spaced layers of a fine-tuned teacher. Each layer $l \\in L'$ is copied fully from $L$. For example, when creating a [BART](https://paperswithcode.com/method/bart) student with 3 decoder layers from the 12 encoder layer 12 decoder layer teacher, we copy the teacher’s full $Enc^{L}$ and decoder layers 0, 6, and 11 to the student. When deciding which layers to copy, we break ties arbitrarily; copying layers 0, 5, and 11 might work just as well. When copy only 1 decoder layer, we copy layer 0. This was found this to work better than copying layer 11. The impact of initialization on performance is measured experimentally in Section 6.1. After initialization, the student model continues to fine-tune on the summarization dataset, with the objective of minimizing $\\mathcal{L}\\_{Data}$.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Pre-trained Summarization Distillation","paper":"/paper/pre-trained-summarization-distillation","first_author":"Sam Shleifer","n_authors":2,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/pre-trained-summarization-distillation"},"source":{"url":"https://arxiv.org/abs/2010.13002v2","title":"Pre-trained Summarization Distillation","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Distillation","url":"/methods/category/distillation","pwc_aliases":[]},{"area":"General","area_id":"general","collection":"Knowledge Distillation","url":"/methods/category/knowledge-distillation","pwc_aliases":[]}],"n_papers_tagged":415,"archive_num_papers":415,"papers_newest_first":[{"paper":"/paper/a-practical-two-stage-recipe-for-mathematical","title":"A Practical Two-Stage Recipe for Mathematical LLMs: Maximizing Accuracy with SFT and Efficiency with Reinforcement Learning","date":"2025-07-11","arxiv_id":"2507.08267","n_code_links":1,"syntology":{"ran":0,"of":6,"unverified":6,"pointer_only":6}},{"paper":"/paper/investalign-overcoming-data-scarcity-in","title":"InvestAlign: Overcoming Data Scarcity in Aligning Large Language Models with Investor Decision-Making Processes under Herd Behavior","date":"2025-07-09","arxiv_id":"2507.06528","n_code_links":1,"syntology":null},{"paper":"/paper/autotriton-automatic-triton-programming-with","title":"AutoTriton: Automatic Triton Programming with Reinforcement Learning in LLMs","date":"2025-07-08","arxiv_id":"2507.05687","n_code_links":1,"syntology":null},{"paper":null,"title":"CogniSQL-R1-Zero: Lightweight Reinforced Reasoning for Efficient SQL Generation","date":"2025-07-08","arxiv_id":"2507.06013","n_code_links":0,"syntology":null},{"paper":"/paper/reinforcement-fine-tuning-naturally-mitigates","title":"Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training","date":"2025-07-07","arxiv_id":"2507.05386","n_code_links":0,"syntology":{"ran":3,"of":10,"unverified":7,"pointer_only":10}},{"paper":"/paper/spiral-self-play-on-zero-sum-games","title":"SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning","date":"2025-06-30","arxiv_id":"2506.24119","n_code_links":1,"syntology":{"ran":2,"of":9,"unverified":7,"pointer_only":0}},{"paper":"/paper/complexity-aware-fine-tuning","title":"Complexity-aware fine-tuning","date":"2025-06-26","arxiv_id":"2506.21220","n_code_links":1,"syntology":null},{"paper":"/paper/longwriter-zero-mastering-ultra-long-text","title":"LongWriter-Zero: Mastering Ultra-Long Text Generation via Reinforcement Learning","date":"2025-06-23","arxiv_id":"2506.18841","n_code_links":1,"syntology":{"ran":6,"of":14,"unverified":8,"pointer_only":0}},{"paper":null,"title":"Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection","date":"2025-06-23","arxiv_id":"2506.18245","n_code_links":0,"syntology":null},{"paper":null,"title":"RLRC: Reinforcement Learning-based Recovery for Compressed Vision-Language-Action Models","date":"2025-06-21","arxiv_id":"2506.17639","n_code_links":0,"syntology":null},{"paper":null,"title":"Differentiation-Based Extraction of Proprietary Data from Fine-Tuned LLMs","date":"2025-06-20","arxiv_id":"2506.17353","n_code_links":0,"syntology":null},{"paper":null,"title":"Refine-POI: Reinforcement Fine-Tuned Large Language Models for Next Point-of-Interest Recommendation","date":"2025-06-19","arxiv_id":"2506.21599","n_code_links":0,"syntology":null},{"paper":null,"title":"Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality","date":"2025-06-17","arxiv_id":"2506.14681","n_code_links":0,"syntology":null},{"paper":null,"title":"AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy","date":"2025-06-16","arxiv_id":"2506.13284","n_code_links":0,"syntology":null},{"paper":null,"title":"Ego-R1: Chain-of-Tool-Thought for Ultra-Long Egocentric Video Reasoning","date":"2025-06-16","arxiv_id":"2506.13654","n_code_links":0,"syntology":null},{"paper":"/paper/metis-rise-rl-incentivizes-and-sft-enhances","title":"Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning","date":"2025-06-16","arxiv_id":"2506.13056","n_code_links":1,"syntology":null},{"paper":"/paper/qfft-question-free-fine-tuning-for-adaptive","title":"QFFT, Question-Free Fine-Tuning for Adaptive Reasoning","date":"2025-06-15","arxiv_id":"2506.12860","n_code_links":1,"syntology":{"ran":3,"of":14,"unverified":11,"pointer_only":0}},{"paper":null,"title":"VGR: Visual Grounded Reasoning","date":"2025-06-13","arxiv_id":"2506.11991","n_code_links":0,"syntology":null},{"paper":"/paper/vision-matters-simple-visual-perturbations","title":"Vision Matters: Simple Visual Perturbations Can Boost Multimodal Math Reasoning","date":"2025-06-11","arxiv_id":"2506.09736","n_code_links":1,"syntology":null},{"paper":"/paper/eliciting-fine-tuned-transformer-capabilities","title":"Eliciting Fine-Tuned Transformer Capabilities via Inference-Time Techniques","date":"2025-06-09","arxiv_id":"2506.08060","n_code_links":1,"syntology":null},{"paper":"/paper/learning-what-reinforcement-learning-can-t","title":"Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questions","date":"2025-06-09","arxiv_id":"2506.07527","n_code_links":1,"syntology":null},{"paper":"/paper/nearness-of-neighbors-attention-for","title":"Nearness of Neighbors Attention for Regression in Supervised Finetuning","date":"2025-06-09","arxiv_id":"2506.08139","n_code_links":1,"syntology":null},{"paper":null,"title":"Learning to Clarify by Reinforcement Learning Through Reward-Weighted Fine-Tuning","date":"2025-06-08","arxiv_id":"2506.06964","n_code_links":0,"syntology":null},{"paper":null,"title":"SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes","date":"2025-06-08","arxiv_id":"2506.07245","n_code_links":0,"syntology":null},{"paper":"/paper/writing-rl-advancing-long-form-writing-via","title":"Writing-RL: Advancing Long-form Writing via Adaptive Curriculum Reinforcement Learning","date":"2025-06-06","arxiv_id":"2506.05760","n_code_links":1,"syntology":null},{"paper":"/paper/mint-cot-enabling-interleaved-visual-tokens","title":"MINT-CoT: Enabling Interleaved Visual Tokens in Mathematical Chain-of-Thought Reasoning","date":"2025-06-05","arxiv_id":"2506.05331","n_code_links":1,"syntology":null},{"paper":null,"title":"On the Mechanism of Reasoning Pattern Selection in Reinforcement Learning for Language Models","date":"2025-06-05","arxiv_id":"2506.04695","n_code_links":0,"syntology":null},{"paper":null,"title":"Crowd-SFT: Crowdsourcing for LLM Alignment","date":"2025-06-04","arxiv_id":"2506.04063","n_code_links":0,"syntology":null},{"paper":null,"title":"RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics","date":"2025-06-04","arxiv_id":"2506.04308","n_code_links":0,"syntology":null},{"paper":null,"title":"Corrigibility as a Singular Target: A Vision for Inherently Reliable Foundation Models","date":"2025-06-03","arxiv_id":"2506.03056","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/reinforcement-learning-1","name":"Reinforcement Learning (RL)","papers":73},{"task":"/task/language-modelling","name":"Language Modelling","papers":50},{"task":"/task/language-modeling","name":"Language Modeling","papers":41},{"task":"/task/large-language-model","name":"Large Language Model","papers":35},{"task":"/task/instruction-following","name":"Instruction Following","papers":34},{"task":"/task/question-answering","name":"Question Answering","papers":30},{"task":"/task/math","name":"Math","papers":29},{"task":"/task/diversity","name":"Diversity","papers":23},{"task":"/task/mathematical-reasoning","name":"Mathematical Reasoning","papers":21},{"task":"/task/reinforcement-learning","name":"Reinforcement Learning","papers":20},{"task":"/task/reinforcement-learning-2","name":"reinforcement-learning","papers":20},{"task":"/task/in-context-learning","name":"In-Context Learning","papers":17},{"task":"/task/code-generation","name":"Code Generation","papers":13},{"task":"/task/gsm8k","name":"GSM8K","papers":13},{"task":"/task/retrieval","name":"Retrieval","papers":11},{"task":"/task/rag","name":"RAG","papers":10},{"task":"/task/text-generation","name":"Text Generation","papers":10},{"task":"/task/visual-reasoning","name":"Visual Reasoning","papers":10},{"task":"/task/retrieval-augmented-generation","name":"Retrieval-augmented Generation","papers":9},{"task":"/task/machine-translation","name":"Machine Translation","papers":8}],"tasks_shown":20,"n_tasks":206,"usage_by_year":[{"year":"2020","papers":1},{"year":"2022","papers":3},{"year":"2023","papers":41},{"year":"2024","papers":187},{"year":"2025","papers":183}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/sft"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}