{"url":"/sota/long-tail-learning-on-inaturalist-2018","task":{"name":"Long-tail Learning","url":"/task/long-tail-learning","note":null},"dataset":{"name":"iNaturalist 2018","url":"/dataset/inaturalist"},"category":"Methodology","categories":["Methodology"],"category_note":null,"description":"Long-tailed learning, one of the most challenging problems in visual recognition, aims to train well-performing models from a large number of images that follow a long-tailed class distribution.","description_from":"task","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","rank":"the archive's row order at snapshot; not re-ranked","rows_end_at":"2025-07-28","rows_withheld_as_spam":0,"metric_values":"the archive's strings, untouched"},"metrics":["Top-1 Accuracy"],"metric_direction":{"note":"inferred from the metric name only (the archive records no direction); null = not inferred, chart draws points only","by_metric":{"Top-1 Accuracy":"higher"}},"counts":{"rows":43,"rows_with_code":38,"rows_with_paper_page":43,"rows_dated":43,"rows_using_additional_data":3},"rows":[{"rank_in_archive_order":1,"model":"LIFT (ViT-L/14@336px)","metrics":{"Top-1 Accuracy":"87.4%"},"uses_additional_data":false,"paper_date":"2023-09-18","paper":"/paper/parameter-efficient-long-tailed-recognition","paper_url":"https://arxiv.org/abs/2309.10019v3","paper_title":"Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts","code":"https://github.com/shijxcs/lift","n_code_links":1,"syntology":{"n_ran":5,"n_unverified":3,"n_samples":8,"n_pointer_only_licence":8}},{"rank_in_archive_order":2,"model":"LIFT (ViT-L/14)","metrics":{"Top-1 Accuracy":"85.2%"},"uses_additional_data":false,"paper_date":"2023-09-18","paper":"/paper/parameter-efficient-long-tailed-recognition","paper_url":"https://arxiv.org/abs/2309.10019v3","paper_title":"Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts","code":"https://github.com/shijxcs/lift","n_code_links":1,"syntology":{"n_ran":5,"n_unverified":3,"n_samples":8,"n_pointer_only_licence":8}},{"rank_in_archive_order":3,"model":"GML (ViT-B-16)","metrics":{"Top-1 Accuracy":"82.1%"},"uses_additional_data":true,"paper_date":"2023-05-02","paper":"/paper/long-tailed-recognition-by-mutual-information","paper_url":"https://arxiv.org/abs/2305.01160v3","paper_title":"Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labels","code":"https://github.com/bluecdm/Long-tailed-recognition","n_code_links":1,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":4,"model":"VL-LTR (ViT-B-16)","metrics":{"Top-1 Accuracy":"81.0%"},"uses_additional_data":true,"paper_date":"2021-11-26","paper":"/paper/vl-ltr-learning-class-wise-visual-linguistic","paper_url":"https://arxiv.org/abs/2111.13579v4","paper_title":"VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognition","code":"https://github.com/ChangyaoTian/VL-LTR","n_code_links":1,"syntology":null},{"rank_in_archive_order":5,"model":"LIFT (ViT-B/16)","metrics":{"Top-1 Accuracy":"80.4%"},"uses_additional_data":false,"paper_date":"2023-09-18","paper":"/paper/parameter-efficient-long-tailed-recognition","paper_url":"https://arxiv.org/abs/2309.10019v3","paper_title":"Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts","code":"https://github.com/shijxcs/lift","n_code_links":1,"syntology":{"n_ran":5,"n_unverified":3,"n_samples":8,"n_pointer_only_licence":8}},{"rank_in_archive_order":6,"model":"RAC (ViT-B-16)","metrics":{"Top-1 Accuracy":"80.24%"},"uses_additional_data":false,"paper_date":"2022-02-22","paper":"/paper/retrieval-augmented-classification-for-long","paper_url":"https://arxiv.org/abs/2202.11233v1","paper_title":"Retrieval Augmented Classification for Long-Tail Visual Recognition","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":7,"model":"GPaCo (2-R152)","metrics":{"Top-1 Accuracy":"79.8%"},"uses_additional_data":false,"paper_date":"2022-09-26","paper":"/paper/generalized-parametric-contrastive-learning","paper_url":"https://arxiv.org/abs/2209.12400v2","paper_title":"Generalized Parametric Contrastive Learning","code":"https://github.com/dvlab-research/parametric-contrastive-learning","n_code_links":4,"syntology":null},{"rank_in_archive_order":8,"model":"GPaCo (ResNet-152)","metrics":{"Top-1 Accuracy":"78.1%"},"uses_additional_data":false,"paper_date":"2022-09-26","paper":"/paper/generalized-parametric-contrastive-learning","paper_url":"https://arxiv.org/abs/2209.12400v2","paper_title":"Generalized Parametric Contrastive Learning","code":"https://github.com/dvlab-research/parametric-contrastive-learning","n_code_links":4,"syntology":null},{"rank_in_archive_order":9,"model":"TADE(ResNet-152)","metrics":{"Top-1 Accuracy":"77%"},"uses_additional_data":false,"paper_date":"2021-07-20","paper":"/paper/test-agnostic-long-tailed-recognition-by-test","paper_url":"https://arxiv.org/abs/2107.09249v4","paper_title":"Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition","code":"https://github.com/vanint/sade-agnosticlt","n_code_links":2,"syntology":{"n_ran":4,"n_unverified":2,"n_samples":6,"n_pointer_only_licence":0}},{"rank_in_archive_order":10,"model":"ProCo (ResNet50)","metrics":{"Top-1 Accuracy":"75.8%"},"uses_additional_data":false,"paper_date":"2024-03-11","paper":"/paper/probabilistic-contrastive-learning-for-long","paper_url":"https://arxiv.org/abs/2403.06726v2","paper_title":"Probabilistic Contrastive Learning for Long-Tailed Visual Recognition","code":"https://github.com/leaplabthu/proco","n_code_links":1,"syntology":{"n_ran":6,"n_unverified":3,"n_samples":9,"n_pointer_only_licence":0}},{"rank_in_archive_order":11,"model":"MDCS(Resnet50)","metrics":{"Top-1 Accuracy":"75.6%"},"uses_additional_data":false,"paper_date":"2023-08-19","paper":"/paper/mdcs-more-diverse-experts-with-consistency","paper_url":"https://arxiv.org/abs/2308.09922v2","paper_title":"MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition","code":"https://github.com/fistyee/mdcs","n_code_links":1,"syntology":{"n_ran":7,"n_unverified":1,"n_samples":8,"n_pointer_only_licence":8}},{"rank_in_archive_order":12,"model":"GPaCo (ResNet-50)","metrics":{"Top-1 Accuracy":"75.4%"},"uses_additional_data":false,"paper_date":"2022-09-26","paper":"/paper/generalized-parametric-contrastive-learning","paper_url":"https://arxiv.org/abs/2209.12400v2","paper_title":"Generalized Parametric Contrastive Learning","code":"https://github.com/dvlab-research/parametric-contrastive-learning","n_code_links":4,"syntology":null},{"rank_in_archive_order":13,"model":"CBD-ENS (ResNet-101)","metrics":{"Top-1 Accuracy":"75.3%"},"uses_additional_data":false,"paper_date":"2021-04-12","paper":"/paper/class-balanced-distillation-for-long-tailed","paper_url":"https://arxiv.org/abs/2104.05279v2","paper_title":"Class-Balanced Distillation for Long-Tailed Visual Recognition","code":"https://github.com/google-research/google-research","n_code_links":3,"syntology":null},{"rank_in_archive_order":14,"model":"PaCo(ResNet-152)","metrics":{"Top-1 Accuracy":"75.2%"},"uses_additional_data":false,"paper_date":"2021-07-26","paper":"/paper/parametric-contrastive-learning","paper_url":"https://arxiv.org/abs/2107.12028v2","paper_title":"Parametric Contrastive Learning","code":"https://github.com/dvlab-research/parametric-contrastive-learning","n_code_links":5,"syntology":{"n_ran":1,"n_unverified":2,"n_samples":3,"n_pointer_only_licence":1}},{"rank_in_archive_order":15,"model":"DeiT-LT","metrics":{"Top-1 Accuracy":"75.1%"},"uses_additional_data":false,"paper_date":"2024-04-03","paper":"/paper/deit-lt-distillation-strikes-back-for-vision","paper_url":"https://arxiv.org/abs/2404.02900v1","paper_title":"DeiT-LT Distillation Strikes Back for Vision Transformer Training on Long-Tailed Datasets","code":"https://github.com/val-iisc/DeiT-LT","n_code_links":2,"syntology":null},{"rank_in_archive_order":16,"model":"APA (SE-ResNet-50)","metrics":{"Top-1 Accuracy":"74.8"},"uses_additional_data":false,"paper_date":"2024-07-11","paper":"/paper/adaptive-parametric-activation","paper_url":"https://arxiv.org/abs/2407.08567v2","paper_title":"Adaptive Parametric Activation","code":"https://github.com/kostas1515/aglu","n_code_links":1,"syntology":null},{"rank_in_archive_order":17,"model":"VL-LTR (ResNet-50)","metrics":{"Top-1 Accuracy":"74.6%"},"uses_additional_data":true,"paper_date":"2021-11-26","paper":"/paper/vl-ltr-learning-class-wise-visual-linguistic","paper_url":"https://arxiv.org/abs/2111.13579v4","paper_title":"VL-LTR: Learning Class-wise Visual-Linguistic Representation for Long-Tailed Visual Recognition","code":"https://github.com/ChangyaoTian/VL-LTR","n_code_links":1,"syntology":null},{"rank_in_archive_order":18,"model":"GML (ResNet-50)","metrics":{"Top-1 Accuracy":"74.5%"},"uses_additional_data":false,"paper_date":"2023-05-02","paper":"/paper/long-tailed-recognition-by-mutual-information","paper_url":"https://arxiv.org/abs/2305.01160v3","paper_title":"Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labels","code":"https://github.com/bluecdm/Long-tailed-recognition","n_code_links":1,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":19,"model":"NCL(ResNet-50)","metrics":{"Top-1 Accuracy":"74.2%"},"uses_additional_data":false,"paper_date":"2022-03-29","paper":"/paper/nested-collaborative-learning-for-long-tailed","paper_url":"https://arxiv.org/abs/2203.15359v2","paper_title":"Nested Collaborative Learning for Long-Tailed Visual Recognition","code":"https://github.com/bazinga699/ncl","n_code_links":1,"syntology":{"n_ran":2,"n_unverified":0,"n_samples":2,"n_pointer_only_licence":0}},{"rank_in_archive_order":20,"model":"BatchFormer(ResNet-50, RIDE)","metrics":{"Top-1 Accuracy":"74.1%"},"uses_additional_data":false,"paper_date":"2022-03-03","paper":"/paper/batchformer-learning-to-explore-sample","paper_url":"https://arxiv.org/abs/2203.01522v2","paper_title":"BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning","code":"https://github.com/zhihou7/batchformer","n_code_links":1,"syntology":{"n_ran":2,"n_unverified":0,"n_samples":2,"n_pointer_only_licence":2}},{"rank_in_archive_order":21,"model":"CBD-ENS (ResNet-50)","metrics":{"Top-1 Accuracy":"73.6%"},"uses_additional_data":false,"paper_date":"2021-04-12","paper":"/paper/class-balanced-distillation-for-long-tailed","paper_url":"https://arxiv.org/abs/2104.05279v2","paper_title":"Class-Balanced Distillation for Long-Tailed Visual Recognition","code":"https://github.com/google-research/google-research","n_code_links":3,"syntology":null},{"rank_in_archive_order":22,"model":"RIDE-DiVE","metrics":{"Top-1 Accuracy":"73.44%"},"uses_additional_data":false,"paper_date":"2021-03-28","paper":"/paper/distilling-virtual-examples-for-long-tailed","paper_url":"https://arxiv.org/abs/2103.15042v3","paper_title":"Distilling Virtual Examples for Long-tailed Recognition","code":"https://github.com/yangyucheng000/DiVE","n_code_links":1,"syntology":null},{"rank_in_archive_order":23,"model":"DirMixE","metrics":{"Top-1 Accuracy":"73.21%"},"uses_additional_data":false,"paper_date":"2024-05-13","paper":"/paper/harnessing-hierarchical-label-distribution","paper_url":"https://arxiv.org/abs/2405.07780v1","paper_title":"Harnessing Hierarchical Label Distribution Variations in Test Agnostic Long-tail Recognition","code":"https://github.com/scongl/dirmixe","n_code_links":1,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":1}},{"rank_in_archive_order":24,"model":"TADE","metrics":{"Top-1 Accuracy":"72.9%"},"uses_additional_data":false,"paper_date":"2021-07-20","paper":"/paper/test-agnostic-long-tailed-recognition-by-test","paper_url":"https://arxiv.org/abs/2107.09249v4","paper_title":"Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition","code":"https://github.com/vanint/sade-agnosticlt","n_code_links":2,"syntology":{"n_ran":4,"n_unverified":2,"n_samples":6,"n_pointer_only_licence":0}},{"rank_in_archive_order":25,"model":"ResLT(3 experts)","metrics":{"Top-1 Accuracy":"72.9%"},"uses_additional_data":false,"paper_date":"2021-01-26","paper":"/paper/reslt-residual-learning-for-long-tailed","paper_url":"https://arxiv.org/abs/2101.10633v3","paper_title":"ResLT: Residual Learning for Long-tailed Recognition","code":"https://github.com/dvlab-research/parametric-contrastive-learning","n_code_links":5,"syntology":{"n_ran":5,"n_unverified":4,"n_samples":9,"n_pointer_only_licence":1}},{"rank_in_archive_order":26,"model":"RIDE","metrics":{"Top-1 Accuracy":"72.2%"},"uses_additional_data":false,"paper_date":"2020-10-05","paper":"/paper/long-tailed-recognition-by-routing-diverse-1","paper_url":"https://arxiv.org/abs/2010.01809v4","paper_title":"Long-tailed Recognition by Routing Diverse Distribution-Aware Experts","code":"https://github.com/frank-xwang/RIDE-LongTailRecognition","n_code_links":2,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":27,"model":"BCL(ResNet-50)","metrics":{"Top-1 Accuracy":"71.8%"},"uses_additional_data":false,"paper_date":"2022-07-19","paper":"/paper/balanced-contrastive-learning-for-long-tailed-1","paper_url":"https://arxiv.org/abs/2207.09052v3","paper_title":"Balanced Contrastive Learning for Long-Tailed Visual Recognition","code":"https://github.com/flamiezhu/bcl","n_code_links":1,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":28,"model":"DiVE","metrics":{"Top-1 Accuracy":"71.71%"},"uses_additional_data":false,"paper_date":"2021-03-28","paper":"/paper/distilling-virtual-examples-for-long-tailed","paper_url":"https://arxiv.org/abs/2103.15042v3","paper_title":"Distilling Virtual Examples for Long-tailed Recognition","code":"https://github.com/yangyucheng000/DiVE","n_code_links":1,"syntology":null},{"rank_in_archive_order":29,"model":"MiSLAS","metrics":{"Top-1 Accuracy":"71.6%"},"uses_additional_data":false,"paper_date":"2021-04-01","paper":"/paper/improving-calibration-for-long-tailed-1","paper_url":"https://arxiv.org/abs/2104.00466v1","paper_title":"Improving Calibration for Long-Tailed Recognition","code":"https://github.com/Jia-Research-Lab/MiSLAS","n_code_links":5,"syntology":{"n_ran":5,"n_unverified":4,"n_samples":9,"n_pointer_only_licence":0}},{"rank_in_archive_order":30,"model":"DisAlign","metrics":{"Top-1 Accuracy":"70.6%"},"uses_additional_data":false,"paper_date":"2021-03-30","paper":"/paper/distribution-alignment-a-unified-framework","paper_url":"https://arxiv.org/abs/2103.16370v1","paper_title":"Distribution Alignment: A Unified Framework for Long-tail Visual Recognition","code":"https://github.com/Megvii-BaseDetection/DisAlign","n_code_links":1,"syntology":{"n_ran":1,"n_unverified":2,"n_samples":3,"n_pointer_only_licence":3}},{"rank_in_archive_order":31,"model":"LDAM-DRS-RSG","metrics":{"Top-1 Accuracy":"70.3%"},"uses_additional_data":false,"paper_date":"2021-06-18","paper":"/paper/rsg-a-simple-but-effective-module-for","paper_url":"https://arxiv.org/abs/2106.09859v1","paper_title":"RSG: A Simple but Effective Module for Learning Imbalanced Datasets","code":"https://github.com/Jianf-Wang/RSG","n_code_links":1,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":32,"model":"IEM","metrics":{"Top-1 Accuracy":"70.2"},"uses_additional_data":false,"paper_date":"2020-06-01","paper":"/paper/inflated-episodic-memory-with-region-self","paper_url":"http://openaccess.thecvf.com/content_CVPR_2020/html/Zhu_Inflated_Episodic_Memory_With_Region_Self-Attention_for_Long-Tailed_Visual_Recognition_CVPR_2020_paper.html","paper_title":"Inflated Episodic Memory With Region Self-Attention for Long-Tailed Visual Recognition","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":33,"model":"LTR-weight-balancing(ResNet-50)","metrics":{"Top-1 Accuracy":"70.2%"},"uses_additional_data":false,"paper_date":"2022-03-27","paper":"/paper/long-tailed-recognition-via-weight-balancing","paper_url":"https://arxiv.org/abs/2203.14197v1","paper_title":"Long-Tailed Recognition via Weight Balancing","code":"https://github.com/shadealsha/ltr-weight-balancing","n_code_links":2,"syntology":{"n_ran":5,"n_unverified":9,"n_samples":14,"n_pointer_only_licence":13}},{"rank_in_archive_order":34,"model":"LDAM + DRW + SAM","metrics":{"Top-1 Accuracy":"70.1"},"uses_additional_data":false,"paper_date":"2022-12-28","paper":"/paper/escaping-saddle-points-for-effective","paper_url":"https://arxiv.org/abs/2212.13827v1","paper_title":"Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data","code":"https://github.com/val-iisc/saddle-longtail","n_code_links":1,"syntology":null},{"rank_in_archive_order":35,"model":"LADE","metrics":{"Top-1 Accuracy":"70.0%"},"uses_additional_data":false,"paper_date":"2020-12-01","paper":"/paper/disentangling-label-distribution-for-long","paper_url":"https://arxiv.org/abs/2012.00321v2","paper_title":"Disentangling Label Distribution for Long-tailed Visual Recognition","code":"https://github.com/hyperconnect/LADE","n_code_links":2,"syntology":null},{"rank_in_archive_order":36,"model":"TSC(ResNet-50)","metrics":{"Top-1 Accuracy":"69.7%"},"uses_additional_data":false,"paper_date":"2021-11-27","paper":"/paper/targeted-supervised-contrastive-learning-for","paper_url":"https://arxiv.org/abs/2111.13998v2","paper_title":"Targeted Supervised Contrastive Learning for Long-Tailed Recognition","code":"https://github.com/lth14/targeted-supcon","n_code_links":1,"syntology":{"n_ran":6,"n_unverified":8,"n_samples":14,"n_pointer_only_licence":0}},{"rank_in_archive_order":37,"model":"CB-LWS","metrics":{"Top-1 Accuracy":"69.5%"},"uses_additional_data":false,"paper_date":"2019-10-21","paper":"/paper/decoupling-representation-and-classifier-for","paper_url":"https://arxiv.org/abs/1910.09217v2","paper_title":"Decoupling Representation and Classifier for Long-Tailed Recognition","code":"https://github.com/facebookresearch/classifier-balancing","n_code_links":4,"syntology":{"n_ran":2,"n_unverified":0,"n_samples":2,"n_pointer_only_licence":2}},{"rank_in_archive_order":38,"model":"MetaSAug","metrics":{"Top-1 Accuracy":"68.75%"},"uses_additional_data":false,"paper_date":"2021-03-23","paper":"/paper/metasaug-meta-semantic-augmentation-for-long","paper_url":"https://arxiv.org/abs/2103.12579v3","paper_title":"MetaSAug: Meta Semantic Augmentation for Long-Tailed Visual Recognition","code":"https://github.com/BIT-DA/MetaSAug","n_code_links":1,"syntology":{"n_ran":2,"n_unverified":1,"n_samples":3,"n_pointer_only_licence":0}},{"rank_in_archive_order":39,"model":"cRT + SSP","metrics":{"Top-1 Accuracy":"68.1%"},"uses_additional_data":false,"paper_date":"2020-06-13","paper":"/paper/rethinking-the-value-of-labels-for-improving","paper_url":"https://arxiv.org/abs/2006.07529v2","paper_title":"Rethinking the Value of Labels for Improving Class-Imbalanced Learning","code":"https://github.com/YyzHarry/imbalanced-semi-self","n_code_links":1,"syntology":{"n_ran":0,"n_unverified":1,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":40,"model":"Hybrid-PSC","metrics":{"Top-1 Accuracy":"68.1%"},"uses_additional_data":false,"paper_date":"2021-03-26","paper":"/paper/contrastive-learning-based-hybrid-networks","paper_url":"https://arxiv.org/abs/2103.14267v1","paper_title":"Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":41,"model":"CE-DRW-IC","metrics":{"Top-1 Accuracy":"67.9%"},"uses_additional_data":false,"paper_date":"2020-10-22","paper":"/paper/posterior-re-calibration-for-imbalanced","paper_url":"https://arxiv.org/abs/2010.11820v1","paper_title":"Posterior Re-calibration for Imbalanced Datasets","code":null,"n_code_links":0,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":42,"model":"Domain Adaptation","metrics":{"Top-1 Accuracy":"67.5%"},"uses_additional_data":false,"paper_date":"2020-03-24","paper":"/paper/rethinking-class-balanced-methods-for-long","paper_url":"https://arxiv.org/abs/2003.10780v1","paper_title":"Rethinking Class-Balanced Methods for Long-Tailed Visual Recognition from a Domain Adaptation Perspective","code":"https://github.com/abdullahjamal/Longtail_DA","n_code_links":1,"syntology":{"n_ran":2,"n_unverified":1,"n_samples":3,"n_pointer_only_licence":0}},{"rank_in_archive_order":43,"model":"Online Feature Augmentation","metrics":{"Top-1 Accuracy":"65.9%"},"uses_additional_data":false,"paper_date":"2020-08-09","paper":"/paper/feature-space-augmentation-for-long-tailed","paper_url":"https://arxiv.org/abs/2008.03673v1","paper_title":"Feature Space Augmentation for Long-Tailed Data","code":null,"n_code_links":0,"syntology":null}],"since_archive":{"claim":"Results that newer papers report for their own method, placed here by Syntology. A model pointed at the cell in the paper's own table; the number was read from that cell and checked against this leaderboard's metric, dataset, split and scale; an independent check that saw this leaderboard's other rows and every other leaderboard on the same dataset accepted it. Not reviewed by the paper's authors or by the archive's editors, and not ranked against the archive rows.","extraction_file_present":true,"measurement":{"test_papers":883,"papers_with_output":881,"judged_true":108,"judged":110,"wilson95_lower":0.9361,"measured_on":"2026-09-24","frozen_commit":"0e3de0df94"},"measurement_note":"blind adjudication of accepted entries on a held-out split of archive papers, rules frozen before the test","coverage":{"sentence":"Syntology has checked 6,885 of the 9,623 papers on this site that are newer than the archive; results from the others appear after they are checked.","complete":false,"papers_newer_than_archive":9623,"papers_checked":6885,"papers_extracted_not_yet_verified":0,"boards_without_verdict":2,"papers_not_yet_extracted":2737},"order":"newest first by month (arXiv date, else the arXiv-id month), then arXiv id descending","columns":[],"entries":[]},"syntology":{"read_at":"2026-09-25T09:33:49+00:00","claim":"Per row: N of M harvested code samples from that row's paper executed on a synthesized fixture; the other M-N are unverified. Not a reproduction of the row's number; not a correctness claim. n_pointer_only_licence counts samples the site points at rather than redistributes (a licence axis, independent of ran/unverified).","rows_with_graph_line":26,"rows_with_any_sample_ran":25,"distinct_papers_with_graph_line":22,"distinct_papers_with_any_sample_ran":21,"samples_over_distinct_papers":{"n_ran":61,"n_unverified":41,"n_samples":102,"n_pointer_only_licence":39,"note":"each paper (arXiv id) counted once, however many rows it is behind; this is the page-level figure"},"samples_row_weighted":{"n_ran":76,"n_unverified":49,"n_samples":125,"n_pointer_only_licence":55,"note":"row-weighted: a paper behind several rows is counted once per row; inflated relative to samples_over_distinct_papers by design, kept for readers summing the per-row syntology blocks"}}}