Large Language Model

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

10 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

3 subtasks in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 2,250 papers with code (6,097 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
  9 Jun 2023 11 repositories listed Syntology ran 0 of 12 samples · 12 unverified
  Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences.
• Generative Agents: Interactive Simulacra of Human Behavior
  7 Apr 2023 8 repositories listed Syntology ran 5 of 12 samples · 7 unverified
  Believable proxies of human behavior can empower interactive applications ranging from immersive environments to rehearsal spaces for interpersonal communication to prototyping tools.
• CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
  25 Mar 2022 8 repositories listed Syntology ran 10 of 10 samples · 0 unverified
  To democratize this, we train and release a family of large language models up to 16.
• HybridFlow: A Flexible and Efficient RLHF Framework
  28 Sep 2024 7 repositories listed Syntology ran 9 of 16 samples · 7 unverified
  Traditional RL can be modeled as a dataflow, where each node represents computation of a neural network (NN) and each edge denotes data dependencies between the NNs.
• Efficient Memory Management for Large Language Model Serving with PagedAttention
  12 Sep 2023 7 repositories listed Syntology ran 25 of 36 samples · 11 unverified
  On top of it, we build vLLM, an LLM serving system that achieves (1) near-zero waste in KV cache memory and (2) flexible sharing of KV cache within and across requests to further reduce memory usage.
• Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model
  4 Jan 2024 6 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  EoH represents the ideas of heuristics in natural language, termed thoughts.
• Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
  16 Nov 2023 6 repositories listed Syntology ran 4 of 7 samples · 3 unverified · 1 pointer-only (licence)
  In this work, we unify visual representation into the language feature space to advance the foundational LLM towards a unified LVLM.
• DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models
  25 Sep 2023 6 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  Computation in a typical Transformer-based large language model (LLM) can be characterized by batch size, hidden dimension, number of layers, and sequence length.
• MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
  20 Apr 2023 6 repositories listed
  Our work, for the first time, uncovers that properly aligning the visual features with an advanced large language model can possess numerous advanced multi-modal abilities demonstrated by GPT-4, such as detailed image…
• A Preview of XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL
  13 Nov 2024 5 repositories listed
  On the other hand, we implement the ICL approach with an example selection method based on named entity recognition to prevent overemphasis on entities.
• LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem
  6 Dec 2023 5 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  We envision that LLM's impact will not be limited to the AI application level, instead, it will in turn revolutionize the design and implementation of computer system, architecture, software, and programming language,…
• Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Following
  1 Sep 2023 5 repositories listed Syntology ran 13 of 20 samples · 7 unverified · 7 pointer-only (licence)
  We introduce Point-Bind, a 3D multi-modality model aligning point clouds with 2D image, language, audio, and video.
• Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data
  3 Apr 2023 5 repositories listed Syntology ran 1 of 9 samples · 8 unverified · 1 pointer-only (licence)
  Furthermore, we propose a new technique called Self-Distill with Feedback, to further improve the performance of the Baize models with feedback from ChatGPT.
• Accelerating Large Language Model Decoding with Speculative Sampling
  2 Feb 2023 5 repositories listed
  We present speculative sampling, an algorithm for accelerating transformer decoding by enabling the generation of multiple tokens from each transformer call.
• Muse: Text-To-Image Generation via Masked Generative Transformers
  2 Jan 2023 5 repositories listed Syntology ran 18 of 21 samples · 3 unverified · 9 pointer-only (licence)
  Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models,…
• A-MEM: Agentic Memory for LLM Agents
  17 Feb 2025 4 repositories listed Syntology ran 6 of 10 samples · 4 unverified
  To address this limitation, this paper proposes a novel agentic memory system for LLM agents that can dynamically organize memories in an agentic way.
• Scaling Synthetic Data Creation with 1,000,000,000 Personas
  28 Jun 2024 4 repositories listed Syntology ran 1 of 4 samples · 3 unverified · 1 pointer-only (licence)
  We propose a novel persona-driven data synthesis methodology that leverages various perspectives within a large language model (LLM) to create diverse synthetic data.
• Ranking Manipulation for Conversational Search Engines
  5 Jun 2024 4 repositories listed Syntology ran 12 of 20 samples · 8 unverified · 20 pointer-only (licence)
  These conversational search engines operate by loading retrieved website text into the LLM context for summarization and interpretation.
• QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
  7 May 2024 4 repositories listed
  The key insight driving QServe is that the efficiency of LLM serving on GPUs is critically influenced by operations on low-throughput CUDA cores.
• Tower: An Open Multilingual Large Language Model for Translation-Related Tasks
  27 Feb 2024 4 repositories listed Syntology ran 7 of 7 samples · 0 unverified · 7 pointer-only (licence)
  While general-purpose large language models (LLMs) demonstrate proficiency on multiple tasks within the domain of translation, approaches based on open LLMs are competitive only when specializing on a single task.
• X-LoRA: Mixture of Low-Rank Adapter Experts, a Flexible Framework for Large Language Models with Applications in Protein Mechanics and Molecular Design
  11 Feb 2024 4 repositories listed
  Starting with a set of pre-trained LoRA adapters, our gating strategy uses the hidden states to dynamically mix adapted layers, allowing the resulting X-LoRA model to draw upon different capabilities and create…
• Algorithm Evolution Using Large Language Model
  26 Nov 2023 4 repositories listed
  In this paper, we propose a novel approach called Algorithm Evolution using Large Language Model (AEL).
• NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmark
  27 Oct 2023 4 repositories listed
  In this position paper, we argue that the classical evaluation on Natural Language Processing (NLP) tasks using annotated benchmarks is in trouble.
• Llemma: An Open Language Model For Mathematics
  16 Oct 2023 4 repositories listed Syntology ran 6 of 8 samples · 2 unverified
  We present Llemma, a large language model for mathematics.
• How is ChatGPT's behavior changing over time?
  18 Jul 2023 4 repositories listed Syntology ran 2 of 6 samples · 4 unverified · 5 pointer-only (licence)
  We find that the performance and behavior of both GPT-3.
• HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution
  27 Jun 2023 4 repositories listed Syntology ran 17 of 28 samples · 11 unverified · 1 pointer-only (licence)
  Leveraging Hyena's new long-range capabilities, we present HyenaDNA, a genomic foundation model pretrained on the human reference genome with context lengths of up to 1 million tokens at the single nucleotide-level - an…
• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
  23 Jun 2023 4 repositories listed
  Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image.
• DeepTextMark: A Deep Learning-Driven Text Watermarking Approach for Identifying Large Language Model Generated Text
  9 May 2023 4 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  As elaborated within the paper, these attributes are crucial for universal text source detection, with a particular emphasis in this paper on text produced by LLMs.
• Protoformer: Embedding Prototypes for Transformers
  25 Jun 2022 4 repositories listed
  This paper proposes Protoformer, a novel self-learning framework for Transformers that can leverage problematic samples for text classification.
• Fast Transformer Decoding: One Write-Head is All You Need
  6 Nov 2019 4 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  Multi-head attention layers, as used in the Transformer neural sequence model, are a powerful alternative to RNNs for moving information across and between sequences.

Syntology lines on 21 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.