Papers › Token-Efficient Leverage Learning in Large Language Models

Token-Efficient Leverage Learning in Large Language Models

1 Apr 2024arXiv:2404.00914archive 2025-07-28

Yuanhao Zeng, Min Wang, Yihang Wang, Yingxia Shao

Large Language Models (LLMs) have excelled in various tasks but perform better in high-resource scenarios, which presents challenges in low-resource scenarios. Data scarcity and the inherent difficulty of adapting LLMs to specific tasks compound the challenge. To address the twin hurdles, we introduce Leverage Learning. We present a streamlined implement of this methodology called Token-Efficient Leverage Learning (TELL). TELL showcases the potential of Leverage Learning, demonstrating effectiveness across various LLMs and low-resource tasks, ranging from 10⁴ to 10⁶ tokens. It reduces task data requirements by up to nearly an order of magnitude compared to conventional Supervised Fine-Tuning (SFT) while delivering competitive performance. With the same amount of task data, TELL leads in improving task performance compared to SFT. We discuss the mechanism of Leverage Learning, suggesting it aligns with quantization hypothesis and explore its promising potential through empirical testing.

PaperPDFCode

Syntology This paper's arXiv id is not in Syntology's Atlas id list (549,449 ids, as this build read them; the read time is in the build record), so this page links to no Syntology page for it. The list is not the whole graph; Syntology's feed shows papers as they enter the graph.

Code

LinesHogan/Token-Efficient-Leverage-Learning officialmentioned on GitHubpytorchSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Instruction FollowingTranslation

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Introduced by this paper: Leverage Learning

Leverage LearningSFT

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections