Browse State-of-the-Art › HellaSwag
HellaSwag
21 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 39 · with a code link: 21 · where Syntology ran a sample: 8 (7 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (8 of 39 tagged: 7 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 39 of 39, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
8 Dec 2021 3 repositories listedLanguage modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
-
29 Mar 2022 2 repositories listed Syntology 8 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 4 pointer-only (licence)We investigate the optimal model size and number of tokens for training a transformer language model under a given compute budget.
-
19 May 2019 2 repositories listed Syntology 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 4 pointer-only (licence)In this paper, we show that commonsense inference still proves difficult for even state-of-the-art models, by presenting HellaSwag, a new challenge dataset.
-
30 May 2025 1 repository listedThe performance of large language models (LLMs) continues to improve, as reflected in rising scores on standard benchmarks.
-
15 Apr 2025 1 repository listed Syntology 11 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 1 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples) · 1 pointer-only (licence)Because large language models are expensive to pretrain on different datasets, using smaller-scale experiments to decide on data is crucial for reducing costs.
-
10 Apr 2025 1 repository listedMeasuring common-sense reasoning, therefore, is crucial for language models of different sizes and applications.
-
13 Jan 2025 1 repository listedTo test the impact of our filtering, we train GPT-2 models on both the original and the filtered datasets.
-
27 Oct 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)Low-rank adaption (LoRA) is a widely used parameter-efficient finetuning method for LLM that reduces memory requirements.
-
6 Sep 2024 1 repository listedThe LayerNorm (LN) layer in GPT-style transformer models has long been a hindrance to mechanistic interpretability.
-
4 Jul 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples) · 9 pointer-only (licence)Large Language Models (LLMs) vary in their abilities on a range of tasks.
-
25 Apr 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)We present LayerSkip, an end-to-end solution to speed-up inference of large language models (LLMs).
-
21 Feb 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)To this extent, we construct the Moral Consistency Corpus (MCC), containing 50K moral questions, responses to them by LLMs, and the RoTs that these models followed.
-
19 Feb 2024 1 repository listedGraphs are commonly used to model complex networks prevalent in modern social media and literacy applications.
-
29 Dec 2023 1 repository listedTo address this gap, our study undertakes a thorough evaluation of Gemini's performance in complex reasoning tasks that necessitate the integration of commonsense knowledge across modalities.
-
26 Oct 2023 1 repository listedWe also introduce an open-source pipeline that enables the community to perform contamination analysis on customised data and models.
-
13 Sep 2023 1 repository listedWe show that, using CrowsPairs dataset, our textual preambles covering counterfactual statements can suppress gender biases in English LLMs such as LLaMA2.
-
8 May 2023 1 repository listedBased on the observation, we propose simple yet effective Contextualized representation-Adversarial Training (CreAT), in which the attack is explicitly optimized to deviate the contextualized representation of the…
-
1 Oct 2022 1 repository listedRecent advances in commonsense reasoning have been fueled by the availability of large-scale human annotated datasets.
-
1 Jul 2022 1 repository listedHence, we examine the effect of a human-like easy-to-difficult curriculum during finetuning of language models for commonsense reasoning tasks.
-
17 Mar 2022 1 repository listedFrom a pre-generated pool of augmented samples, Glitter adaptively selects a subset of worst-case samples with maximal loss, analogous to adversarial DA.
-
24 Mar 2021 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples)First, we propose a new multitask benchmark, RAINBOW, to promote research on commonsense models that generalize well over multiple tasks and datasets.
-
Slimming Down LLMs Without Losing Their Minds12 Jun 2025 0 repositories listedThis paper investigates and validates the impact of fine-tuning on large language model performance, focusing on parameter-efficient methods (LoRA and QLoRA).
-
Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst20 May 2025 0 repositories listedInference-time scaling has attracted much attention which significantly enhance the performance of Large Language Models (LLMs) in complex reasoning tasks by increasing the length of Chain-of-Thought.
-
Elastic Weight Consolidation for Full-Parameter Continual Pre-Training of Gemma29 May 2025 0 repositories listedThis technical report describes an experiment on autoregressive pre-training of Gemma2 2 billion parameter large language model (LLM) with 10% on the Lithuanian language component of CulturaX from the point of view of…
-
Domain-Adaptive Continued Pre-Training of Small Language Models13 Apr 2025 0 repositories listedContinued pre-training of small language models offers a promising path for domain adaptation with limited computational resources.
-
More is Less: The Pitfalls of Multi-Model Synthetic Preference Data in DPO Safety Alignment3 Apr 2025 0 repositories listedAligning large language models (LLMs) with human values is an increasingly critical step in post-training.
-
Obliviate: Efficient Unmemorization for Protecting Intellectual Property in Large Language Models20 Feb 2025 0 repositories listedRecent copyright agreements between AI companies and content creators underscore the need for fine-grained control over language models' ability to reproduce copyrighted text.
-
HellaSwag-Pro: A Large-Scale Bilingual Benchmark for Evaluating the Robustness of LLMs in Commonsense Reasoning17 Feb 2025 0 repositories listedLarge language models (LLMs) have shown remarkable capabilities in commonsense reasoning; however, some variations in questions can trigger incorrect responses.
-
SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better Reasoning in SLMs11 Dec 2024 0 repositories listedWe present SmolTulu-1.7b-Instruct, referenced in this report as SmolTulu-DPO-1130, an instruction-tuned language model that adapts AllenAI's Tulu 3 post-training pipeline to enhance Huggingface's SmolLM2-1.7B base model.
-
Towards Multilingual LLM Evaluation for European Languages11 Oct 2024 0 repositories listedThe rise of Large Language Models (LLMs) has revolutionized natural language processing across numerous languages and tasks.
-
Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs30 Sep 2024 0 repositories listed
-
GRIN: GRadient-INformed MoE18 Sep 2024 0 repositories listed
-
Promises, Outlooks and Challenges of Diffusion Language Modeling17 Jun 2024 0 repositories listed
-
Who's Harry Potter? Approximate Unlearning in LLMs3 Oct 2023 0 repositories listed
-
Contrastive Decoding Improves Reasoning in Large Language Models17 Sep 2023 0 repositories listed
-
When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation16 Nov 2021 0 repositories listed
-
Comparing Test Sets with Item Response Theory1 Jun 2021 0 repositories listed
-
26 May 2020 0 repositories listed
-
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning29 Apr 2020 0 repositories listed
Syntology lines on 8 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections