Browse State-of-the-Art › Long-Context Understanding
Long-Context Understanding
54 papers with code · 5 benchmarks · 4 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
5 leaderboard tables shown for this task, 5 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology 10 newer papers were judged to report on this task's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 1 placed): LongBench (10 judged, 1 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
4 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 81 · with a code link: 54 · where Syntology ran a sample: 28 (21 with a run with no instrument failure, 7 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (28 of 81 tagged: 21 with a run with no instrument failure, 7 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 81 of 81, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
9 Jun 2023 11 repositories listed Syntology community repositories only · 9 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 12 harvested samples)Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences.
-
15 Mar 2023 11 repositories listed Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 1 pointer-only (licence)We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs.
-
5 Oct 2022 9 repositories listed Syntology official (archive's flag): 9 ran · 15 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 0 honoured, 0 violated, 14 with no contract checked; 1 where Syntology's instrument failed) · 6 unverified (of 21 harvested samples)We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters.
-
9 Apr 2024 6 repositories listed Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)Despite achieving nearly perfect accuracy in the vanilla NIAH test, almost all models exhibit large performance drops as the context length increases.
-
9 Dec 2024 4 repositories listed Syntology official (archive's flag): 5 ran · 11 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 2 honoured, 0 violated, 6 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples) · 7 pointer-only (licence)Linear Transformers have gained attention as efficient alternatives to standard Transformers, but their performance in retrieval and long-context tasks has been limited.
-
6 Nov 2023 4 repositories listedWe introduce CogVLM, a powerful open-source visual language foundation model.
-
11 May 2023 4 repositories listedLarge-scale pre-training and instruction tuning have been successful at creating general-purpose language models with broad competence.
-
1 Apr 2024 3 repositories listed Syntology community repositories only · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)While LLM-based auto-raters have proven reliable for factuality and coherence in other settings, we implement several LLM raters of faithfulness and find that none correlates strongly with human annotations, especially…
-
26 Mar 2024 3 repositories listed Syntology 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
-
28 Aug 2023 3 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)In this paper, we introduce LongBench, the first bilingual, multi-task benchmark for long context understanding, enabling a more rigorous evaluation of long context understanding.
-
3 Oct 2024 2 repositories listedLong-context models (LCMs) have made remarkable strides in recent years, offering users great convenience for handling tasks that involve long context, such as document summarization.
-
2 Apr 2024 2 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples)We introduce a benchmark (LongICLBench) for long in-context learning in extreme-label classification using six datasets with 28 to 174 classes and input lengths from 2K to 50K tokens.
-
7 Nov 2023 2 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples) · 3 pointer-only (licence)Multi-modal Large Language Models (MLLMs) have demonstrated impressive instruction abilities across various open-ended tasks.
-
23 Oct 2023 2 repositories listed Syntology official (archive's flag): 21 ran · 21 ran (of which 0 constructed an object rather than computing a result; 21 with no instrument failure: 0 honoured, 0 violated, 21 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 26 harvested samples) · 26 pointer-only (licence)The rapid development of Large Language Models (LLMs) has led to great strides in model capabilities like long-context understanding and reasoning.
-
13 Jul 2025 1 repository listedLong-context language models (LCLMs) have exhibited impressive capabilities in long-context understanding tasks.
-
20 Jun 2025 1 repository listedIn this paper, we propose the *KV footprint* as a unified metric, which accounts for both the amount of KV entries stored and their lifespan in memory.
-
6 Jun 2025 1 repository listedLong-context understanding is crucial for many NLP applications, yet transformers struggle with efficiency due to the quadratic complexity of self-attention.
-
5 Jun 2025 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)Sequence modeling is currently dominated by causal transformer architectures that use softmax self-attention.
-
27 May 2025 1 repository listedSpeculative decoding is a widely adopted technique for accelerating inference in large language models (LLMs), but its performance degrades on long inputs due to increased attention cost and reduced draft accuracy.
-
26 May 2025 1 repository listed Syntology official (archive's flag): 2 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 2 pointer-only (licence)Post-training compression reduces the computational and memory costs of large language models (LLMs), enabling resource-efficient deployment.
-
26 May 2025 1 repository listedLong Context Understanding (LCU) is a critical area for exploration in current large language models (LLMs).
-
20 May 2025 1 repository listedAlthough the context length of large language models (LLMs) has increased to millions of tokens, evaluating their effectiveness beyond needle-in-a-haystack approaches has proven difficult.
-
24 Apr 2025 1 repository listedLong-context understanding poses significant challenges in natural language processing, particularly for real-world dialogues characterized by speech-based elements, high redundancy, and uneven information density.
-
LongMamba: Enhancing Mamba's Long Context Capabilities via Training-Free Receptive Field Enlargement22 Apr 2025 1 repository listedState space models (SSMs) have emerged as an efficient alternative to Transformer models for language modeling, offering linear computational complexity and constant memory usage as context length increases.
-
10 Apr 2025 1 repository listedWe present Kimi-VL, an efficient open-source Mixture-of-Experts (MoE) vision-language model (VLM) that offers advanced multimodal reasoning, long-context understanding, and strong agent capabilities - all while…
-
14 Mar 2025 1 repository listedWe introduce CURIE, a scientific long-Context Understanding, Reasoning and Information Extraction benchmark to measure the potential of Large Language Models (LLMs) in scientific problem-solving and assisting scientists…
-
12 Mar 2025 1 repository listedWe evaluate all methods on multiple long-context understanding tasks and show that non-uniform loss weights are helpful to improve the long-context abilities of LLMs.
-
11 Mar 2025 1 repository listed Syntology official (archive's flag): 5 ran · 7 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 9 unverified (of 16 harvested samples) · 16 pointer-only (licence)Recently, Probabilistic Language-Image Pre-Training (ProLIP) has been proposed to tackle the multiplicity issue of vision-language (VL) tasks.
-
21 Feb 2025 1 repository listedAnswering complex, long-context questions remains a major challenge for large language models (LLMs) as it requires effective question clarifications and context retrieval.
-
19 Feb 2025 1 repository listedEvaluating large language models' (LLMs) long-context understanding capabilities remains challenging.
-
12 Feb 2025 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
11 Feb 2025 1 repository listed
-
4 Feb 2025 1 repository listed
-
1 Feb 2025 1 repository listed Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 2 pointer-only (licence)
-
12 Dec 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified (of 10 harvested samples)
-
31 Oct 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)
-
21 Oct 2024 1 repository listed
-
20 Oct 2024 1 repository listed
-
24 Sep 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
28 Jun 2024 1 repository listed Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
25 Jun 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 2 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 8 harvested samples)
-
21 Jun 2024 1 repository listed Syntology official (archive's flag): 19 ran · 19 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 9 where Syntology's instrument failed) · 3 unverified (of 22 harvested samples)
-
17 Jun 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
10 Jun 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
4 Jun 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
23 May 2024 1 repository listed
-
24 Apr 2024 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
16 Apr 2024 1 repository listed Syntology official (archive's flag): 12 ran · 12 ran (of which 1 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 0 violated, 6 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified (of 15 harvested samples) · 15 pointer-only (licence)
-
9 Apr 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
18 Mar 2024 1 repository listed
-
8 Mar 2024 1 repository listed
-
19 Dec 2023 1 repository listed
-
15 Dec 2023 1 repository listed
-
8 Nov 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)
-
PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding18 Jun 2025 0 repositories listed
-
ATLAS: Learning to Optimally Memorize the Context at Test Time29 May 2025 0 repositories listed
-
Beyond Needle(s) in the Embodied Haystack: Environment, Architecture, and Training Considerations for Long Context Reasoning22 May 2025 0 repositories listed
-
Towards Robust Evaluation of STEM Education: Leveraging MLLMs in Project-Based Learning16 May 2025 0 repositories listed
-
MOOSComp: Improving Lightweight Long-Context Compressor via Mitigating Over-Smoothing and Incorporating Outlier Scores23 Apr 2025 0 repositories listed
-
LIFT: Improving Long Context Understanding of Large Language Models through Long Input Fine-Tuning20 Feb 2025 0 repositories listed
-
Facilitating Long Context Understanding via Supervised Chain-of-Thought Reasoning18 Feb 2025 0 repositories listed
-
Can LLMs Maintain Fundamental Abilities under KV Cache Compression?4 Feb 2025 0 repositories listed
-
Guided Code Generation with LLMs: A Multi-Agent Framework for Complex Code Tasks11 Jan 2025 0 repositories listed
-
Repository Structure-Aware Training Makes SLMs Better Issue Resolver26 Dec 2024 0 repositories listed
-
LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning18 Dec 2024 0 repositories listed
-
State Space Models are Strong Text Rerankers18 Dec 2024 0 repositories listed
-
How Effective Is Self-Consistency for Long-Context Problems?2 Nov 2024 0 repositories listed
-
Enhancing Scientific Reproducibility Through Automated BioCompute Object Creation Using Retrieval-Augmented Generation from Publications23 Sep 2024 0 repositories listed
-
E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning10 Sep 2024 0 repositories listed
-
Retrieval Or Holistic Understanding? Dolce: Differentiate Our Long Context Evaluation Tasks10 Sep 2024 0 repositories listed
-
Large Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement Learning4 Sep 2024 0 repositories listed
-
ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities19 Jul 2024 0 repositories listed
-
Fine-Tuning Medical Language Models for Enhanced Long-Contextual Understanding and Domain Expertise16 Jul 2024 0 repositories listed
-
Anomaly Detection of Tabular Data Using LLMs24 Jun 2024 0 repositories listed
-
Equipping Transformer with Random-Access Reading for Long-Context Understanding21 May 2024 0 repositories listed
-
3 May 2024 0 repositories listed
-
XL²Bench: A Benchmark for Extremely Long Context Understanding with Long-range Dependencies8 Apr 2024 0 repositories listed
-
4 Mar 2024 0 repositories listed
-
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models3 Feb 2024 0 repositories listed
-
24 Jul 2023 0 repositories listed
-
Revisiting Parallel Context Windows: A Frustratingly Simple Alternative and Chain-of-Thought Deterioration24 May 2023 0 repositories listed
Syntology lines on 29 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections