Home › Code › extract_answer

extract_answer

Syntologyentry name in harvested coderead from the graph 2026-09-24

extract_answer appears in the code Syntology harvested for 112 papers, as 112 distinct code bodies found in 126 places (a place is one code body under one paper). At least one of them ran in 68 of the papers; 59 of the code bodies carry a behaviour fingerprint.

What this page is not. Routines are grouped here by the exact string of their function or class name. Nothing asserts that two samples named extract_answer do the same thing, share code, or are comparable; the name is a string, not an identity. Behaviour outputs (what a fingerprinted sample returned on the shared battery) are not in this export and are not shown here; the graph at syntology.ai holds them. "Ran" means executed on a synthesized fixture, not that the code is correct or reproduces a paper.

Samples Syntology

Syntology ran 66 of the 112 distinct code bodies named extract_answer; 46 are unverified. One tile per status, in the site's fixed vocabulary, each code body counted once:

1ran · honoured contract
0ran · violated contract
26ran · our draft was wrong
0ran · fixture could not drive it
39ran
46unverified
59fingerprinted

Licence is a property of each copy, so it is counted per place: 55 of the 126 places are pointer only (Syntology does not serve that copy's text). This site shows no code text for any sample; every row below links to the file in its repository where the record names one.

“Ran” means the sample executed on a synthesized input; it does not mean the output is correct. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code, and those samples did run. The ran count above is every status except unverified, the same rule as each paper page.

Papers

100 papers shown of 112 (newest first; the JSON twin carries all 126 places); 112 places in the table. A paper with no recorded date is placed by the month its arXiv id encodes, shown in the Date column as YYYY-MM (from id). One row per place: a paper whose repository defines the name more than once appears more than once, and the same code body held for several papers appears once under each, with the same status. Titles and dates are the archive's archive 2025-07-28 for papers in the archive, and the graph's for 33 papers added by Syntology; 6 papers have no page here and are shown by arXiv id only. Status and fingerprint are Syntology's record of each code body; licence is recorded for each place. The File cell ends with the code body's code_sha256, Syntology's identity for that exact code: an agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

PaperDateFileStatus SyntologyLicence
GTA: Graph Theory Agent and Benchmark for Algorithmic Graph Reasoning with LLMs added by Syntology 2026-09 (from id) xzx34/GTA/src/gtbench/evaluate.py 0a5cb941be6403a6 unverified MIT (permissive)
Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding added by Syntology 2026-09 (from id) MinooAhmadii/chopthin-consensus-power-sampling/ccps/graders/answers.py 31f2eef78b769944 unverified licence not identified · pointer only
Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It added by Syntology 2026-08 (from id) ngqm/belief-fact-phrasing/src/eval/instruction_effect_table.py 520d0858be37dec1 ran fingerprinted MIT (permissive)
CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits added by Syntology 2026-08 (from id) mehrshad-sdtn/CircuitSteer/experiments/gsm8k.py 038ceb4e56fe6031 ran fingerprinted no licence file found · pointer only
Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes added by Syntology 2026-07 (from id) ZhouYuxuanYX/Fast-HSD/fast_hsd/benchmarks/_math_scoring.py 69e80a64efad173b ran fingerprinted Apache-2.0 (permissive)
A Gradient Perspective on RLVR Stability and Winner Advantage Policy Optimization added by Syntology 2026-06 (from id) layer6ai-labs/wapo/environments/math_dapo/math_dapo.py f3be998c40770890 ran fingerprinted MIT (permissive)
A Gradient Perspective on RLVR Stability and Winner Advantage Policy Optimization added by Syntology 2026-06 (from id) layer6ai-labs/wapo/environments/math_prm/math_prm.py 059412997faa2f93 ran fingerprinted MIT (permissive)
Sandboxed Coding Agents are Competitive Omni-modal Task Solvers added by Syntology 2026-06 (from id) Dongping-Chen/OmniCoding/src/omnicoding/rl/reward.py 51e9b7e8d7ad73f8 ran fingerprinted licence not identified · pointer only
Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement added by Syntology 2026-05 (from id) CuSO4-Chen/AKBE/AKBE/verl_akbe/verl/utils/reward_score/reward_em_betagrpo.py 8aec27dfaeff217d ran · our draft was wrong fingerprinted no licence file found · pointer only
Reasoning, Code, or Both? How Large Language Models Handle Variations in Math Questions added by Syntology 2026-05 (from id) masamodelkin/llm-robustness-code-execution/src/evals/CoT.py f8f102d5cfc1883c ran fingerprinted MIT (permissive)
Can LLMs Time Travel? Enhancing Temporal Consistency in Legal Agentic Search through Reinforcement Learning added by Syntology 2026-05 (from id) AlexFanw/LegalSearch-R1/user/calculate_metrics.py 04f6d2711d4cd42e ran fingerprinted Apache-2.0 (permissive)
Published at ICLR 2026 Workshop on Reasoning and Planning for LLMs RESIDUAL DRIFT DOMINATES CONTRADICTION IN MULTI-TURN CONSTRAINT REASONING added by Syntology 2026-05 (from id) kaons-research/drift-bench/src/extraction.py 2859b0d4274869b4 unverified MIT (permissive)
Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models added by Syntology 2026-05 (from id) giovanni-vaccarino/PUMA/puma/extract_final_candidates.py 000a843cb1f68ddb ran · our draft was wrong no licence file found · pointer only
MASPO: Joint Prompt Optimization for LLM-based Multi-Agent Systems added by Syntology 2026-05 (from id) wangzx1219/MASPO/utils.py 8ae4dd0f7d0bba2f ran fingerprinted no licence file found · pointer only
Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking added by Syntology 2026-05 (from id) jznmsl/siren/03_analysis_scripts/study_e_multimodel_api.py 99767d2dbf24d8cf ran fingerprinted MIT (permissive)
Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking added by Syntology 2026-05 (from id) jznmsl/siren/03_analysis_scripts/study_e_multimodel_law.py 16e4d0149abd853f ran fingerprinted MIT (permissive)
Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking added by Syntology 2026-05 (from id) jznmsl/siren/03_analysis_scripts/build_tensor.py bec005b22dd07b60 ran fingerprinted MIT (permissive)
FinSTaR: Towards Financial Reasoning with Time Series Reasoning Models added by Syntology 2026-05 (from id) seunghan96/FinSTaR/src/evaluation/utils.py ed37ec1880683750 ran fingerprinted licence not identified · pointer only
Do Vision-Language Models Truly Perform Vision Reasoning? A Rigorous Study of the Modality Gap added by Syntology 2026-04 (from id) xuyige/CrossMath/batch_inference_qwen35.py c53d52049bcc871f unverified licence not identified · pointer only
Doc-V * : Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA added by Syntology 2026-04 (from id) SeerRay-Lab/Doc-V/inference/agent.py 52a7f587ad7bd317 unverified no licence file found · pointer only
Doc-V * : Coarse-to-Fine Interactive Visual Reasoning for Multi-Page Document VQA added by Syntology 2026-04 (from id) SeerRay-Lab/Doc-V/inference/rag.py 2dcb4cc46acab4c7 unverified no licence file found · pointer only
Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions added by Syntology 2026-04 (from id) zhangwenhao6/PERA/eval_commonsense.py d14efab97e468115 unverified licence not identified · pointer only
A Systematic Analysis of the Impact of Persona Steering on LLM Capabilities added by Syntology 2026-04 (from id) cjia7/DPR/src/npti/eval/eval_mmlu.py 0ad16da4e4f52ed7 unverified no licence file found · pointer only
The Format Tax added by Syntology 2026-04 (from id) ivnle/the-format-tax/formats/freeform.py d093f593ac440765 unverified licence not identified · pointer only
LIE TO ME: HOW FAITHFUL IS CHAIN-OF-THOUGHT REASONING IN OPEN-WEIGHT REASONING MODELS? added by Syntology 2026-03 (from id) ricyoung/cot-faithfulness-open-models/src/cot_faithfulness/answer_extraction.py 8735a844d15f3a2a unverified licence not identified · pointer only
LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards added by Syntology 2026-03 (from id) real-absolute-AI/LongRLVR/recipe/dapo/longrl_reward_manager.py b9fa0381260e409d ran · our draft was wrong fingerprinted no licence file found · pointer only
Multimodal Fact-Level Attribution for Verifiable Reasoning added by Syntology 2026-02 (from id) meetdavidwan/murgat/src/util.py ca419c78d00ec1d5 unverified MIT (permissive)
Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection added by Syntology 2026-02 (from id) SunGL001/OGPSA/eval/GPQA_Diamond.py a7b81f2f5969d9ab unverified MIT (permissive)
Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection added by Syntology 2026-02 (from id) SunGL001/OGPSA/eval/MMLU.py b9ce5a37c260fee5 unverified MIT (permissive)
BranPO: Scalable Contrastive Branch Sampling for Long-Horizon Agentic Reinforcement Learning added by Syntology 2026-02 (from id) YubaoZhao/BranPO/agent/search/llm_as_a_judge.py ec1077be0c2df0e6 unverified Apache-2.0 (permissive)
When Agents "Misremember" Collectively: Exploring the Mandela Effect in LLM-based Multi-Agent Systems added by Syntology 2026-02 (from id) bluedream02/Mandela-Effect/eval_correct.py 1f183be69c67626e ran · our draft was wrong fingerprinted MIT (permissive)
Reasoning While Asking: Transforming Reasoning Large Language Models from Passive Solvers to Proactive Inquirers added by Syntology 2026-01 (from id) SUAT-AIRI/Proactive-Interactive-R1/generalization_eval/factual_knowledge/run_mmlu_interactive_generation.py adf4a17f6030f2ef unverified Apache-2.0 (permissive)
Advances in LLM Reasoning Enable Flexibility in Clinical Problem-Solving added by Syntology 2026-01 (from id) bernardolab/mARC-Reasoning/compute_accuracy.py 1f59ab74392c3fcf ran · our draft was wrong fingerprinted Apache-2.0 (permissive)
Skeletons Matter: Dynamic Data Augmentation for Text-to-Query added by Syntology 2025-11 (from id) jjjycaptain/Skeletron/src/data_syn.py 85b5611a1add5852 ran · our draft was wrong fingerprinted no licence file found · pointer only
Teaching Language Models to Reason with Tools added by Syntology 2025-10 (from id) ChengpengLi1003/CoRT/infer/parser.py 951c55f8667c735f unverified no licence file found · pointer only
SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning added by Syntology 2025-10 (from id) PKU-ML/SSL4RL/verl/utils/reward_score/ssl4rl.py 93c07e2e0da6f774 ran · our draft was wrong fingerprinted Apache-2.0 (permissive)
mmWalk: Towards Multi-modal Multi-view Walking Assistance added by Syntology 2025-10 (from id) KediYing/mmWalk/eval_gpt.py 22f07d287593a1b8 ran · our draft was wrong no licence file found · pointer only
ConfTuner: Training Large Language Models to Express Their Confidence Verbally added by Syntology 2025-08 (from id) liushiliushi/ConfTuner/src/llama_recipes/datasets2/gsm8k_dataset.py d95f990735eba145 ran · our draft was wrong fingerprinted no licence file found · pointer only
arXiv:2507.02834 2025-07 (from id) dhcode-cpp/X-R1/src/x_r1/rewards.py ebe018f39b3bb7d5 unverified Apache-2.0 (permissive)
WebSailor: Navigating Super-human Reasoning for Web Agent 3 Jul 2025 alibaba-nlp/webwalker/evaluation/evaluate_hle_official.py 079a31f347eec894 unverified Apache-2.0 (permissive)
PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models 25 Jun 2025 soufiane001/plop/sft/eval_gsm8k.py 54d8e73534679cb9 unverified no licence file found · pointer only
ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs 23 Jun 2025 yangling0818/buffer-of-thought-llm/meta_buffer_utilis.py 92d7b41138b8da5b unverified MIT (permissive)
arXiv:2506.18105 2025-06 (from id) sofyc/ChengyuBench/run-appropriateness.py c7bbfd05d008d535 unverified no licence file found · pointer only
arXiv:2506.18105 2025-06 (from id) sofyc/ChengyuBench/run-connotation.py e035e8b22820fb1a unverified no licence file found · pointer only
How Far Can LLMs Improve from Experience? Measuring Test-Time Learning Ability in LLMs with Human Comparison 17 Jun 2025 Alice1998/Test-time-Learning/cumulative_setting/dynamic_cheatsheet/utils/extractor.py ef43d275e410158b unverified no licence file found · pointer only
DiaBlo: Diagonal Blocks Are Sufficient For Finetuning 3 Jun 2025 ziyangjoy/diablo/evaluate_commonsense.py 95f77b72bcc7ef90 unverified no licence file found · pointer only
DeepTheorem: Advancing LLM Reasoning for Theorem Proving Through Natural Language and Reinforcement Learning 29 May 2025 jiahao004/deeptheorem/eval/eval_llm.py 2b9cebd93558867a ran · honoured contract fingerprinted no licence file found · pointer only
ToMAP: Training Opponent-Aware LLM Persuaders with Theory of Mind 29 May 2025 ulab-uiuc/ToMAP/verl/env_feedback/argument_graph.py 4a1bf7c49ac2d0c9 unverified Apache-2.0 (permissive)
Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space 19 May 2025 bigai-nlco/latentseek/src/extract_judge_answer/utils.py 6b13e7afc2042384 unverified MIT (permissive)
Multi-Token Prediction Needs Registers 15 May 2025 nasosger/mutor/language_modeling/src/eval/evaluate_gsm8k.py 0e4c30e9e6691a25 unverified MIT (permissive)
Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving 12 May 2025 yyht/openrlhf_async_pipline/evaluation/my_evaluation.py 4986aab3cd8d8212 ran · our draft was wrong fingerprinted Apache-2.0 (permissive)
DeepCritic: Deliberate Critique with Large Language Models 1 May 2025 rucbm/deepcritic/Critique_Generation/gen_step_solutions.py 315054dfedd692ff ran · our draft was wrong fingerprinted MIT (permissive)
Adaptive Orchestration of Modular Generative Information Access Systems 24 Apr 2025 informagi/AQA/AQA_dataset_organizer.py 767285e542cae380 unverified MIT (permissive)
Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models 25 Mar 2025 stogiannidis/srbench/src/eval/acc.py 14a27694a27f4ad0 unverified MIT (permissive)
Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuning 20 Mar 2025 minglllli/cls-rl/src/eval/test_cvbench.py 2bb2ff42bd737b97 ran · our draft was wrong fingerprinted no licence file found · pointer only
RepoST: Scalable Repository-Level Coding Environment Construction with Sandbox Testing 10 Mar 2025 yiqingxyq/RepoST/RepoST/llm_check.py 34dcaa09d95a53cd unverified MIT (permissive)
Large Language Models Often Say One Thing and Do Another 10 Mar 2025 icip-cas/Word-Deed-Consistency-Test/eval/test_local_models.py 4ca8f0e08bcf3036 ran · our draft was wrong fingerprinted no licence file found · pointer only
arXiv:2503.05188 2025-03 (from id) BugMakerzzz/CRISP/crisp_reason.py 82ee443d76f92d28 ran · our draft was wrong fingerprinted no licence file found · pointer only
R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model 7 Mar 2025 identical code first harvested elsewhere 2bb2ff42bd737b97 ran · our draft was wrong fingerprinted licence of this copy not recorded
BIG-Bench Extra Hard 26 Feb 2025 google-deepmind/bbeh/bbeh/evaluate.py 1148b66bf0848e3d ran · our draft was wrong fingerprinted Apache-2.0 (permissive)
Exploring the Limit of Outcome Reward for Learning Mathematical Reasoning 10 Feb 2025 internlm/oreal/oreal/judgers/utils.py 0e8f4b09eb5b8c47 unverified Apache-2.0 (permissive)
Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling 10 Feb 2025 RyanLiu112/compute-optimal-tts/src/envs/MATH/env.py 042094830d54a879 unverified MIT (permissive)
STAIR: Improving Safety Alignment with Introspective Reasoning 4 Feb 2025 thu-ml/stair/src/final_orm.py fcfc0ecd7309f71a unverified MIT (permissive)
Do as We Do, Not as You Think: the Conformity of Large Language Models 23 Jan 2025 Zhiyuan-Weng/BenchForm/reflection.py 1f183be69c67626e ran · our draft was wrong fingerprinted MIT (permissive)
VoxEval: Benchmarking the Knowledge Understanding Capabilities of End-to-End Spoken Language Models 9 Jan 2025 dreamtheater123/voxeval/metric_calculation.py 95d8414ac0a018a0 ran · our draft was wrong fingerprinted no licence file found · pointer only
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks 19 Dec 2024 identical code first harvested elsewhere 2807cfec4fc4d41e ran · our draft was wrong fingerprinted licence of this copy not recorded
LLaVA-UHD v2: an MLLM Integrating High-Resolution Feature Pyramid via Hierarchical Window Transformer 18 Dec 2024 thunlp/llava-uhd/VLMEvalKit/vlmeval/vlm/llava_qwen2_uhd_v3.py a76fe5da7ade58fb unverified Apache-2.0 (permissive)
HARP: A challenging human-annotated math reasoning benchmark 11 Dec 2024 aadityasingh/harp/src/eval/parsing_lib.py cf5843ce85c71a69 unverified MIT (permissive)
ProcessBench: Identifying Process Errors in Mathematical Reasoning 9 Dec 2024 qwenlm/processbench/code/run_eval.py 315054dfedd692ff ran · our draft was wrong fingerprinted no licence file found · pointer only
R-CoT: Reverse Chain-of-Thought Problem Generation for Geometric Reasoning in Large Multimodal Models 23 Oct 2024 dle666/r-cot/MathVista_eval/evaluation/extract_answer.py c1eaa5bf4d5f383d unverified no licence file found · pointer only
InternLM2.5-StepProver: Advancing Automated Theorem Proving via Expert Iteration on Large-Scale LEAN Problems 21 Oct 2024 internlm/internlm-math/agent/math_agent.py 6cd811d1cce3b5e7 unverified Apache-2.0 (permissive)
Rationale-Aware Answer Verification by Pairwise Self-Evaluation 7 Oct 2024 akirakawabata/reps/src/reward_training.py 747f8df0daf7cfe2 ran MIT (permissive)
SORSA: Singular Values and Orthonormal Regularized Singular Vectors Adaptation of Large Language Models 21 Aug 2024 Gunale0926/SORSA/dataset.py c923d19564d2546f ran fingerprinted Apache-2.0 (permissive)
The Llama 3 Herd of Models 31 Jul 2024 wenet-e2e/west/west/bin/decode_mmau.py 0a0d3ec6fb8f3c46 unverified Apache-2.0 (permissive)
The Llama 3 Herd of Models 31 Jul 2024 wenet-e2e/west/west/bin/decode_mmsu.py d7b280c2f396affc unverified Apache-2.0 (permissive)
Harmonizing Visual Text Comprehension and Generation 23 Jul 2024 bytedance/textharmony/TextHarmony/utils/vqa_score.py c1b06375bdb265f4 ran fingerprinted Apache-2.0 (permissive)
Scaling Retrieval-Based Language Models with a Trillion-Token Datastore 9 Jul 2024 rulinshao/retrieval-scaling/src/evaluate_perplexity.py 290c5134d15244b3 ran fingerprinted MIT (permissive)
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases 17 Jul 2024 BillChan226/AgentPoison/ReAct/search.py fcc271edc48aad1c ran MIT (permissive)
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks 26 Jun 2024 dmg-illc/JUDGE-BENCH/eval_responses.py 4203c0dc81afe24e ran · our draft was wrong fingerprinted no licence file found · pointer only
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs 24 Jun 2024 kiddyboots216/lottery-ticket-adaptation/rlaif/eval_model_all.py bd0cc71c62b4a212 ran fingerprinted Apache-2.0 (permissive)
Can Large Language Models Always Solve Easy Problems if They Can Solve Harder Ones? 18 Jun 2024 QwenLM/ConsisEval/math_check/check.py b22b268ce5bb5b68 ran fingerprinted MIT (permissive)
Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge 12 Jun 2024 Slimshilin/Position-Bias-Analyzer-Demo/position_bias_analyzer/calculate_consistency_and_preference/consistency_preference_util.py 726da9cc7c6c575a ran no licence file found · pointer only
Husky: A Unified, Open-Source Language Agent for Multi-Step Reasoning 10 Jun 2024 agent-husky/husky-v1/husky/run_husky.py bf89f824c7dab150 ran · our draft was wrong fingerprinted no licence file found · pointer only
Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models 6 Jun 2024 YangLing0818/buffer-of-thought-llm/meta_buffer_utilis.py 92d7b41138b8da5b unverified MIT (permissive)
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark 3 Jun 2024 tiger-ai-lab/mmlu-pro/compute_accuracy.py 1f59ab74392c3fcf ran · our draft was wrong fingerprinted Apache-2.0 (permissive)
Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training 31 May 2024 calubkk/RAAT/tuner/utils/answer_processor.py fa90936b1f466162 ran fingerprinted no licence file found · pointer only
DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception 24 May 2024 rainbowluocs/deem/uni_interleaved/utils/vqa_score.py c1b06375bdb265f4 ran fingerprinted Apache-2.0 (permissive)
From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step 23 May 2024 da03/internalize_cot_step_by_step/src/data.py 4d589052839e4a5e ran fingerprinted MIT (permissive)
LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Study 26 Apr 2024 aix-group/llms-for-cfs/src/gen_cf/generate_hatespeech.py ed21c4ca2697e0d9 ran fingerprinted no licence file found · pointer only
LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Study 26 Apr 2024 aix-group/llms-for-cfs/src/gen_cf/generate_imdb.py be81c1789179c1ac ran fingerprinted no licence file found · pointer only
Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewards 16 Apr 2024 hbin0701/Self-Explore/gen/utils_others.py 50408bcb3e87455b ran fingerprinted no licence file found · pointer only
DyKnow: Dynamically Verifying Time-Sensitive Factual Knowledge in LLMs 10 Apr 2024 sislab-unitn/dyknow/models_output/analyze_replies.py 17600052c8ca574e ran MIT (permissive)
Chain-of-Action: Faithful and Multimodal Question Answering through Large Language Models 26 Mar 2024 MAGICS-LAB/Chain-of-Actions/chain-of-search-wo-ir.py 047a7d86ce36d383 ran fingerprinted Apache-2.0 (permissive)
Tapilot-Crossing: Benchmarking and Evolving LLMs Towards Interactive Data Analysis Agents 8 Mar 2024 tapilot-crossing/tapilot_code/eval/eval_multi_choice.py 8c5a9f85651f57df ran fingerprinted MIT (permissive)
Common 7B Language Models Already Possess Strong Math Capabilities 7 Mar 2024 jerrywu-code/susgen/eval/code/eval_finqa.py 30e951ff25cd61d7 unverified MIT (permissive)
Case-Based or Rule-Based: How Do Transformers Do the Math? 27 Feb 2024 graphpku/case_or_rule/datasets/addition/cot_gen.py fd391d2d2217abf1 ran fingerprinted MIT (permissive)
ToMBench: Benchmarking Theory of Mind in Large Language Models 23 Feb 2024 zhchen18/tombench/get_results.py 048418fb06a97048 ran · our draft was wrong fingerprinted MIT (permissive)
Divide-or-Conquer? Which Part Should You Distill Your LLM? 22 Feb 2024 apple/ml-divide-or-conquer/evaluation/evaluation.py 2c7f4d404b0a3c5b ran fingerprinted licence not identified · pointer only
Divide-or-Conquer? Which Part Should You Distill Your LLM? 22 Feb 2024 apple/ml-divide-or-conquer/evaluation/evaluation_bamb.py 9efb8ade48dbb583 ran fingerprinted licence not identified · pointer only
OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems 21 Feb 2024 OpenBMB/OlympiadBench/inference/judge.py 5328984c0ec57f93 ran MIT (permissive)
BBox-Adapter: Lightweight Adapting for Black-Box Large Language Models 13 Feb 2024 haotiansun14/bbox-adapter/utils/gsm8k_metric.py 6731183eb0c6819d ran fingerprinted no licence file found · pointer only
A Thorough Examination of Decoding Methods in the Era of LLMs 10 Feb 2024 davidfanzz/llm_decoding/lm_eval/tasks/gsm8k.py ac208e27f559cfcb unverified no licence file found · pointer only
Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding 23 Jan 2024 suzgunmirac/meta-prompting/evaluate_outputs.py 58a6661987e8b91b unverified MIT (permissive)
Linear Alignment: A Closed-form Solution for Aligning Human Preferences without Tuning and Feedback 21 Jan 2024 wizardcoast/linear_alignment/preference_eval.py b4b74510d16a049c ran · our draft was wrong fingerprinted MIT (permissive)
Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning 19 Jan 2024 Yiwei98/ESC/consistency_gsm8k.py d95f990735eba145 ran · our draft was wrong fingerprinted no licence file found · pointer only
Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning 19 Jan 2024 Yiwei98/ESC/consistency_coin.py 6ec18dd15818d034 ran fingerprinted no licence file found · pointer only
Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning 19 Jan 2024 Yiwei98/ESC/consistency_csqa.py b550af4a369be974 ran fingerprinted no licence file found · pointer only
Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning 19 Jan 2024 Yiwei98/ESC/consistency_last.py 691cea85ad41fe0e ran fingerprinted no licence file found · pointer only
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding 4 Dec 2023 lntzm/cvpr24track-longvideo/benchmark/evaluate_egoschema.py 8c3317746d989d00 unverified BSD-3-Clause (permissive)
PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning 15 Nov 2023 ytyz1307zzh/plug/src/evaluate/svamp/evaluate_svamp.py 3e78d709f22fd30c ran fingerprinted no licence file found · pointer only
Implicit Chain of Thought Reasoning via Knowledge Distillation 2 Nov 2023 da03/implicit_chain_of_thought/src/data.py 4d589052839e4a5e ran fingerprinted no licence file found · pointer only
Online Speculative Decoding 11 Oct 2023 liuxiaoxuanpku/osd/distill/data.py d95f990735eba145 ran · our draft was wrong fingerprinted no licence file found · pointer only

This site shows no code text; each File cell links to the file on GitHub at the repository's current default branch, which may have changed since the harvest. "Pointer only" means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence cell for the reason. Per-sample records for a paper are on its paper page under "Code Syntology ran".

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections