Papers › ChainDoRA: Tensor-Train Factorized Weight-Decomposed Low-Rank Adaptation for...

ChainDoRA: Tensor-Train Factorized Weight-Decomposed Low-Rank Adaptation for Parameter-Efficient LLM Fine-Tuning

8 Sep 2026arXiv:2609.25058added by Syntology

Ashfak Yeafi, Mehedi Hasan, Md Khairul Islam

Title, abstract, authors and date from arXiv's metadata (CC0); this paper is not in the Papers with Code archive (frozen 2025-07-28).

Parameter-efficient fine-tuning (PEFT) adapts large language models (LLMs) to downstream tasks while updating only a small fraction of their pretrained parameters. Low-Rank Adaptation (LoRA) uses two trainable low-rank matrices, while Weight-Decomposed Low-Rank Adaptation (DoRA) further separates weight magnitude and direction but retains the dense LoRA-style factorization in its directional branch. We propose ChainDoRA, a weight-decomposed adaptation framework that constructs the directional low-rank factors from a connected Tensor-Train (TT) chain, where the adapter rank forms the boundary rank between input- and output-side TT contractions and an independent TT rank controls representation capacity and parameter cost. Under a controlled 15,119-example response-only adaptation setting with LLaMA-7B, ChainDoRA is evaluated against matched LoRA and DoRA baselines on seven commonsense reasoning benchmarks. ChainDoRA with TT rank 16 achieves a seven-task average accuracy of 72.30%, compared with 69.88% for LoRA and 69.39% for DoRA, while requiring only 5.35M trainable parameters versus 56.10M for LoRA and 56.98M for DoRA, corresponding to a 90.62% reduction relative to DoRA. Ablations over TT rank and adapter placement show controllable parameter-accuracy trade-offs, indicating that connected TT parameterization can substantially reduce the parameter cost of magnitude-direction adaptation while preserving, and in this setting improving, downstream reasoning performance.

Code

Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Syntology holds the repository link but has not harvested or run code from it.

Results from the paper

The Papers with Code archive ends with its 2025-07-28 snapshot. This paper's arXiv identifier, 2609.25058, was issued in September 2026, after that date, so the archive has no leaderboard rows for it.

Placed on leaderboards by Syntology Syntology

Syntology's extractor, the model Claude Sonnet 4.5, judged this paper's tables to report on 8 leaderboards (a model's judgement, not a result) and has not placed the paper on any of them: Common Sense Reasoning · ARC (Challenge) (refused by a rule: the table does not name the leaderboard's dataset); Common Sense Reasoning · ARC (Easy) (refused by a rule: the table does not name the leaderboard's dataset); Common Sense Reasoning · WinoGrande (rejected by the independent check); parameter-efficient fine-tuning · BoolQ (rejected by the independent check); parameter-efficient fine-tuning · WinoGrande (rejected by the independent check); Question Answering · BoolQ (rejected by the independent check); Question Answering · OBQA (rejected by the independent check); Question Answering · PIQA (rejected by the independent check). What is not shown.

Syntology has checked 8,886 of the 9,662 papers on this site that are newer than the archive (for 3,717 of them no archive leaderboard matched the paper's tables, so there was nothing further to check); 733 were read and have nothing to place (arXiv has no HTML version of the paper, or that version has no tables), 41 could not be read (the extractor's reply could not be parsed), and results from the other 2 appear after they are checked.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections