Papers › RuleR: Improving LLM Controllability by Rule-based Data Recycling

RuleR: Improving LLM Controllability by Rule-based Data Recycling

22 Jun 2024arXiv:2406.15938archive 2025-07-28

Ming Li, Han Chen, Chenguang Wang, Dang Nguyen, Dianqi Li, Tianyi Zhou

Despite the remarkable advancement of Large language models (LLMs), they still lack delicate controllability under sophisticated constraints, which is critical to enhancing their response quality and the user experience. While conditional supervised fine-tuning (SFT) can potentially improve LLM controllability, curating new SFT data to fulfill the constraints usually relies on human experts or proprietary LLMs, which is time-consuming and expensive. To bridge this gap, we propose Rule-based Data Recycling (RuleR), a human/LLM-free data augmentation method incorporating multiple constraints into the original SFT data. Instead of creating new responses from scratch, RuleR integrates linguistic or formatting rules into the original instructions and modifies the responses to fulfill the rule-defined constraints. Training on the "recycled" data consolidates LLMs capability to generate constrained outputs. Extensive experiments demonstrate RuleR's effectiveness in improving LLM controllability while maintaining general instruction-following performance. RuleR's code is released on https://github.com/tianyi-lab/RuleR.

PaperPDFCode

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

mingliiii/ruler officialmentioned in papermentioned on GitHub report
tianyi-lab/ruler officialmentioned in papermentioned on GitHub report
tianyi-lab/mosaic-it mentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Data AugmentationInstruction Following

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

SFT

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections