parameter-efficient fine-tuning

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 441 papers with code (935 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• LoRA: Low-Rank Adaptation of Large Language Models
  17 Jun 2021 74 repositories listed Syntology ran 34 of 84 samples · 50 unverified · 28 pointer-only (licence)
  We propose Low-Rank Adaptation, or LoRA, which freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, greatly reducing the number of…
• QLoRA: Efficient Finetuning of Quantized LLMs
  23 May 2023 20 repositories listed Syntology ran 17 of 26 samples · 9 unverified · 17 pointer-only (licence)
  Our best model family, which we name Guanaco, outperforms all previous openly released models on the Vicuna benchmark, reaching 99.
• Any2Point: Empowering Any-modality Large Models for Efficient 3D Understanding
  11 Apr 2024 7 repositories listed Syntology ran 15 of 24 samples · 9 unverified · 24 pointer-only (licence)
  The adapter incorporates prior spatial knowledge from the source modality to guide the local feature aggregation of 3D tokens, compelling the semantic adaption of any-modality transformers.
• Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models
  4 Oct 2023 7 repositories listed
  To this end, we introduce Point-PEFT, a novel framework for adapting point cloud pre-trained models with minimal learnable parameters.
• BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
  18 Jun 2021 6 repositories listed Syntology ran 4 of 13 samples · 9 unverified
  We introduce BitFit, a sparse-finetuning method where only the bias-terms of the model (or a subset of them) are being modified.
• DoRA: Weight-Decomposed Low-Rank Adaptation
  14 Feb 2024 5 repositories listed Syntology ran 8 of 15 samples · 7 unverified · 14 pointer-only (licence)
  By employing \ours, we enhance both the learning capacity and training stability of LoRA while avoiding any additional inference overhead.
• Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following
  1 Sep 2023 5 repositories listed Syntology ran 13 of 20 samples · 7 unverified · 7 pointer-only (licence)
  We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image, language, audio, and video.
• DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution
  27 May 2024 4 repositories listed
  Fine-tuning large-scale pre-trained models is inherently a resource-intensive task.
• S-LoRA: Serving Thousands of Concurrent LoRA Adapters
  6 Nov 2023 4 repositories listed
  To capitalize on these opportunities, we present S-LoRA, a system designed for the scalable serving of many LoRA adapters.
• LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
  21 Sep 2023 4 repositories listed Syntology ran 11 of 13 samples · 2 unverified
  For example, training on the context length of 8192 needs 16x computational costs in self-attention layers as that of 2048.
• Balancing LoRA Performance and Efficiency with Simple Shard Sharing
  19 Sep 2024 3 repositories listed
  This method constructs the low-rank update matrix through the replication of these shared, partitioned shards.
• Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentation
  24 Jun 2024 3 repositories listed
  We introduce a new model - Segment any Text (SaT) - to solve this problem.
• PoliTune: Analyzing the Impact of Data Selection and Fine-Tuning on Economic and Political Biases in Large Language Models
  10 Apr 2024 3 repositories listed
  In an era where language models are increasingly integrated into decision-making and communication, understanding the biases within Large Language Models (LLMs) becomes imperative, especially when these models are…
• An Embarrassingly Simple Approach for LLM with Strong ASR Capacity
  13 Feb 2024 3 repositories listed
  We found that delicate designs are not necessary, while an embarrassingly simple composition of off-the-shelf speech encoder, LLM, and the only trainable linear projector is competent for the ASR task.
• Learning to Route Among Specialized Experts for Zero-Shot Generalization
  8 Feb 2024 3 repositories listed Syntology ran 4 of 4 samples · 0 unverified
  Unlike past methods that learn to route among specialized models, PHATGOOSE explores the possibility that zero-shot generalization will be improved if different experts can be adaptively chosen for each token and at…
• When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications
  21 Oct 2023 3 repositories listed Syntology ran 4 of 6 samples · 2 unverified
  To address these two problems, we propose a novel parameter efficient fine-tuning framework for multi-task medical applications, dubbed as MOELoRA.
• OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models
  4 Jun 2023 3 repositories listed Syntology ran 2 of 2 samples · 0 unverified
  Large language models (LLMs) with hundreds of billions of parameters require powerful server-grade GPUs for inference, limiting their practical deployment.
• GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generation
  31 Mar 2023 3 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 1 pointer-only (licence)
  Recent breakthroughs in the field of language-guided image generation have yielded impressive achievements, enabling the creation of high-quality and diverse images based on user instructions.
• TASTE: Text-Aligned Speech Tokenization and Embedding for Spoken Language Modeling
  9 Apr 2025 2 repositories listed Syntology ran 14 of 20 samples · 6 unverified · 20 pointer-only (licence)
  To our knowledge, TASTE is the first end-to-end approach that utilizes a reconstruction objective to automatically learn a text-aligned speech tokenization and embedding suitable for spoken language modeling.
• FarExStance: Explainable Stance Detection for Farsi
  18 Dec 2024 2 repositories listed
  We compare the performance of a fine-tuned multilingual RoBERTa model to several large language models in zero-shot, few-shot, and parameter-efficient fine-tuned settings on our new dataset.
• Prompt Compression for Large Language Models: A Survey
  16 Oct 2024 2 repositories listed
  Leveraging large language models (LLMs) for complex natural language tasks typically requires long-form prompts to convey detailed requirements and information, which results in increased memory usage and inference…
• Parameter Efficient Fine-tuning via Explained Variance Adaptation
  9 Oct 2024 2 repositories listed Syntology ran 3 of 4 samples · 1 unverified
  To this end, we introduce Explained Variance Adaptation (EVA), an initialization scheme that uses the directions capturing the most activation variance, provably maximizing the expected gradient signal and accelerating…
• Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition
  24 Sep 2024 2 repositories listed
  Second, despite similar accuracy, we find that PEFT methods make different mistakes and high-confidence predictions, likely due to their different inductive biases.
• Sam2Rad: A Segmentation Model for Medical Images with Learnable Prompts
  10 Sep 2024 2 repositories listed
  SAM and its variants often fail to segment structures in ultrasound (US) images due to domain shift.
• Soft Language Prompts for Language Transfer
  2 Jul 2024 2 repositories listed
  We further present to our knowledge the first use of soft prompts for language transfer, a technique we call soft language prompts.
• Low-Rank Few-Shot Adaptation of Vision-Language Models
  28 May 2024 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  Recent progress in the few-shot adaptation of Vision-Language Models (VLMs) has further pushed their generalization capabilities, at the expense of just a few labeled samples within the target downstream task.
• IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT
  2 Apr 2024 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  This is also a notable improvement over the Adapter and LoRA, which require 37-39 GB GPU memory and 350-380 seconds per epoch for training.
• Advancing Parameter Efficiency in Fine-tuning via Representation Editing
  23 Feb 2024 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 1 pointer-only (licence)
  Parameter Efficient Fine-Tuning (PEFT) techniques have drawn significant attention due to their ability to yield competitive results while updating only a small portion of the adjustable parameters.
• KInIT at SemEval-2024 Task 8: Fine-tuned LLMs for Multilingual Machine-Generated Text Detection
  21 Feb 2024 2 repositories listed
  SemEval-2024 Task 8 is focused on multigenerator, multidomain, and multilingual black-box machine-generated text detection.
• TuneTables: Context Optimization for Scalable Prior-Data Fitted Networks
  17 Feb 2024 2 repositories listed Syntology ran 10 of 17 samples · 7 unverified
  Notably, TabPFN achieves very strong performance on small tabular datasets but is not designed to make predictions for datasets of size larger than 1000.

Syntology lines on 17 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.