Papers › Chain of Agents: Large Language Models Collaborating on Long-Context Tasks

Chain of Agents: Large Language Models Collaborating on Long-Context Tasks

4 Jun 2024arXiv:2406.02818archive 2025-07-28

Yusen Zhang, Ruoxi Sun, Yanfei Chen, Tomas Pfister, Rui Zhang, Sercan Ö. Arik

Addressing the challenge of effectively processing long contexts has become a critical issue for Large Language Models (LLMs). Two common strategies have emerged: 1) reducing the input length, such as retrieving relevant chunks by Retrieval-Augmented Generation (RAG), and 2) expanding the context window limit of LLMs. However, both strategies have drawbacks: input reduction has no guarantee of covering the part with needed information, while window extension struggles with focusing on the pertinent information for solving the task. To mitigate these limitations, we propose Chain-of-Agents (CoA), a novel framework that harnesses multi-agent collaboration through natural language to enable information aggregation and context reasoning across various LLMs over long-context tasks. CoA consists of multiple worker agents who sequentially communicate to handle different segmented portions of the text, followed by a manager agent who synthesizes these contributions into a coherent final output. CoA processes the entire input by interleaving reading and reasoning, and it mitigates long context focus issues by assigning each agent a short context. We perform comprehensive evaluation of CoA on a wide range of long-context tasks in question answering, summarization, and code completion, demonstrating significant improvements by up to 10% over strong baselines of RAG, Full-Context, and multi-agent LLMs.

Paper PDF

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Code Completion Long-Form Narrative Summarization Question Answering RAG Retrieval-augmented Generation

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Long-Form Narrative Summarization	BookSum	Chain of Agents (Claude 3 Opus)	ROUGE (geometric mean of 1/2/L)	17.47	#7 of 10	Archive leaderboard	report
Long-Form Narrative Summarization	BookSum	Chain of Agents (Claude 3 Sonet)	ROUGE (geometric mean of 1/2/L)	14.96	#9 of 10	Archive leaderboard	report
Long-Form Narrative Summarization	BookSum	Chain of Agents (Claude 3 Haiku)	ROUGE (geometric mean of 1/2/L)	13.70	#10 of 10	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Adam Attention Attention Dropout BART BERT BPE Dense Connections Dropout Focus Layer Normalization Linear Layer Linear Warmup With Linear Decay Multi-Head Attention RAG Residual Connection Softmax Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections