Code Summarization

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

8 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 82 papers with code (163 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• A Transformer-based Approach for Source Code Summarization
  1 May 2020 9 repositories listed
  Generating a readable summary that describes the functionality of a program is known as source code summarization.
• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
  9 Feb 2021 7 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  Benchmark datasets have a significant impact on accelerating research in programming language tasks.
• Recommendations for Datasets for Source Code Summarization
  4 Apr 2019 7 repositories listed
  The main use for these descriptions is in software documentation e.
• code2seq: Generating Sequences from Structured Representations of Code
  4 Aug 2018 6 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  The ability to generate natural language sequences from source code snippets has a variety of applications such as code summarization, documentation, and retrieval.
• CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation
  2 Sep 2021 5 repositories listed Syntology ran 1 of 11 samples · 10 unverified
  We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed from the developer-assigned identifiers.
• CodeT5+: Open Code Large Language Models for Code Understanding and Generation
  13 May 2023 2 repositories listed Syntology ran 3 of 4 samples · 1 unverified
  To address these limitations, we propose ``CodeT5+'', a family of encoder-decoder LLMs for code in which component modules can be flexibly combined to suit a wide range of downstream code tasks.
• Graph Conditioned Sparse-Attention for Improved Source Code Understanding
  1 Dec 2021 2 repositories listed
  The fusion between a graph representation like Abstract Syntax Tree (AST) and a source code sequence makes the use of current approaches computationally intractable for large input sequence lengths.
• Retrieval Augmented Code Generation and Summarization
  26 Aug 2021 2 repositories listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  To mimic developers' code or summary generation behavior, we propose a retrieval augmented framework, REDCODER, that retrieves relevant code or summaries from a retrieval database and provides them as a supplement to…
• HAConvGNN: Hierarchical Attention Based Convolutional Graph Neural Network for Code Documentation Generation in Jupyter Notebooks
  31 Mar 2021 2 repositories listed
  Jupyter notebook allows data scientists to write machine learning code together with its documentation in cells.
• Unified Pre-training for Program Understanding and Generation
  10 Mar 2021 2 repositories listed
  Experiments on code summarization in the English language, code generation, and code translation in seven programming languages show that PLBART outperforms or rivals state-of-the-art models.
• Improved Code Summarization via a Graph Neural Network
  6 Apr 2020 2 repositories listed
  The first approaches to use structural information flattened the AST into a sequence.
• Code Generation as a Dual Task of Code Summarization
  14 Oct 2019 2 repositories listed Syntology ran 4 of 7 samples · 3 unverified · 3 pointer-only (licence)
  Code summarization (CS) and code generation (CG) are two crucial tasks in the field of automatic software development.
• Improving Automatic Source Code Summarization via Deep Reinforcement Learning
  17 Nov 2018 2 repositories listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  To the best of our knowledge, most state-of-the-art approaches follow an encoder-decoder framework which encodes the code into a hidden space and then decode it into natural language space, suffering from two major…
• Variational Prefix Tuning for Diverse and Accurate Code Summarization Using Pre-trained Language Models
  14 May 2025 1 repository listed
  Recent advancements in source code summarization have leveraged transformer-based pre-trained models, including Large Language Models of Code (LLMCs), to automate and improve the generation of code summaries.
• Code Summarization Beyond Function Level
  23 Feb 2025 1 repository listed
  The study involved revising benchmarks for evaluating models at class and repository levels, assessing baseline models, and evaluating LLMs with in-context learning to determine the enhancement of summary quality with…
• How to Select Pre-Trained Code Models for Reuse? A Learning Perspective
  7 Jan 2025 1 repository listed
  Pre-training a language model and then fine-tuning it has shown to be an efficient and effective technique for a wide range of code intelligence tasks, such as code generation, code summarization, and vulnerability…
• Transducer Tuning: Efficient Model Adaptation for Software Tasks Using Code Property Graphs
  18 Dec 2024 1 repository listed
  We introduce \approach, a technique to adapt large models for downstream code tasks using Code Property Graphs (CPGs).
• Model Editing for LLMs4Code: How Far are We?
  11 Nov 2024 1 repository listed
  Despite that, a comprehensive study that thoroughly compares and analyzes the performance of the state-of-the-art model editing techniques for adapting the knowledge within LLMs4Code across various code-related tasks is…
• Building A Coding Assistant via the Retrieval-Augmented Language Model
  21 Oct 2024 1 repository listed
  Specifically, it consists of a code structure aware retriever (CONAN-R) and a dual-view code representation-based retrieval-augmented generation model (CONAN-G).
• XMainframe: A Large Language Model for Mainframe Modernization
  5 Aug 2024 1 repository listed
  To this end, we introduce XMainframe, a state-of-the-art large language model (LLM) specifically designed with knowledge of mainframe legacy systems and COBOL codebases.
• Source Code Summarization in the Era of Large Language Models
  9 Jul 2024 1 repository listed
  We hope that our findings can provide a comprehensive understanding of code summarization in the era of LLMs.
• InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct
  8 Jul 2024 1 repository listed Syntology ran 11 of 11 samples · 0 unverified · 11 pointer-only (licence)
  Recent advancements in open-source code large language models (LLMs) have been driven by fine-tuning on the data generated from powerful closed-source LLMs, which are expensive to obtain.
• AutoCoder: Enhancing Code Large Language Model with \textsc{AIEV-Instruct}
  23 May 2024 1 repository listed
  We introduce AutoCoder, the first Large Language Model to surpass GPT-4 Turbo (April 2024) and GPT-4o in pass@1 on the Human Eval benchmark test (90.
• Analyzing the Performance of Large Language Models on Code Summarization
  10 Apr 2024 1 repository listed
  We show that for the task of code summarization, the performance of these models on individual examples often depends on the amount of (subword) token overlap between the code and the corresponding reference natural…
• CSA-Trans: Code Structure Aware Transformer for AST
  7 Apr 2024 1 repository listed Syntology ran 5 of 5 samples · 0 unverified · 5 pointer-only (licence)
  When applying the Transformer architecture to source code, designing a good self-attention mechanism is critical as it affects how node relationship is extracted from the Abstract Syntax Trees (ASTs) of the source code.
• Empirical Studies of Parameter Efficient Methods for Large Language Models of Code and Knowledge Transfer to R
  16 Mar 2024 1 repository listed
  In this work, we evaluate PEFT methods, LoRA, Compacter, and IA^3 on LLMs for code summarization and generation, with a particular emphasis on knowledge transfer to R as an unseen under-explored target language.
• EyeTrans: Merging Human and Machine Attention for Neural Code Summarization
  21 Feb 2024 1 repository listed
  Neural code summarization leverages deep learning models to automatically generate brief natural language summaries of code snippets.
• Studying Vulnerable Code Entities in R
  6 Feb 2024 1 repository listed
  Pre-trained Code Language Models (Code-PLMs) have shown many advancements and achieved state-of-the-art results for many software engineering tasks in the past few years.
• A Prompt Learning Framework for Source Code Summarization
  26 Dec 2023 1 repository listed
  Our comprehensive experimental results show that PromptCS significantly outperforms instruction prompting schemes (including zero-shot learning and few-shot learning) on all four widely used metrics, and is comparable…
• Binary Code Summarization: Benchmarking ChatGPT/GPT-4 and Other Large Language Models
  15 Dec 2023 1 repository listed
  Binary code summarization, while invaluable for understanding code semantics, is challenging due to its labor-intensive nature.

Syntology lines on 9 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.