Papers › Long-context Language Models Cannot Retrieve Without Sufficient Steps

Long-context Language Models Cannot Retrieve Without Sufficient Steps

6 Oct 2024arXiv:2410.04422archive 2025-07-28

Yijiong Yu, Ma Xiufa, Fang Jianwei, Zhi Xu, Su Guangyao, Wang Jiancheng, Yongfeng Huang, Zhixiao Qi, Wei Wang, Weifeng Liu, Ran Chen, Ji Pei

Long-context language models (LCLMs), characterized by their extensive context window, are becoming popular. However, despite they are nearly perfect at standard long-context retrieval tasks, we find they are not good at all types of retrieval tasks. Specifically, we identify 2 basic cases, "multi-matching retrieval," and "logic-based retrieval", which are beyond LCLMs' ability boundary under normal settings. Later, we find these cases can be well addressed with a specific number of reasoning steps, guided by specific CoT prompts, but it may cost too much time. Thus we propose a critical viewpoint that there are currently no perfect solutions for current LCLMs to solve all types of retrieval tasks. Our work reveals some novel properties of retrieval tasks and LCLMs, proving that long-context handling still has a long way to go.

PaperPDFCode

Code

yuyijiong/hard_retrieval_for_llm officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Retrieval

Datasets

Introduced by this paper, per the archive.

difficult retrieval

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections