{"url":"/sota/natural-language-inference-on-snli","task":{"name":"Natural Language Inference","url":"/task/natural-language-inference","note":null},"dataset":{"name":"SNLI","url":"/dataset/snli"},"category":"Natural Language Processing","categories":["Natural Language Processing","Reasoning"],"category_note":null,"description":"**Natural language inference (NLI)** is the task of determining whether a \"hypothesis\" is \r\ntrue (entailment), false (contradiction), or undetermined (neutral) given a \"premise\".\r\n\r\nExample:\r\n\r\n| Premise | Label | Hypothesis |\r\n| --- | ---| --- |\r\n| A man inspects the uniform of a figure in some East Asian country. | contradiction | The man is sleeping. |\r\n| An older and younger man smiling. | neutral  | Two men are smiling and laughing at the cats playing on the floor. |\r\n| A soccer game with multiple males playing. | entailment | Some men are playing a sport. |\r\n\r\nApproaches used for NLI include earlier symbolic and statistical approaches to more recent deep learning approaches. Benchmark datasets used for NLI include [SNLI](/dataset/snli), [MultiNLI](/dataset/multinli), [SciTail](/dataset/scitail), among others. You can get hands-on practice on the SNLI task by following this [d2l.ai chapter](https://d2l.ai/chapter_natural-language-processing-applications/natural-language-inference-and-dataset.html).\r\n\r\nFurther readings:\r\n\r\n- [Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches](https://arxiv.org/abs/1904.01172)","description_from":"task","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","rank":"the archive's row order at snapshot; not re-ranked","rows_end_at":"2025-07-28","rows_withheld_as_spam":0,"metric_values":"the archive's strings, untouched"},"metrics":["% Test Accuracy","% Train Accuracy","Parameters","Dev Accuracy","% Dev Accuracy","Accuracy"],"metric_direction":{"note":"inferred from the metric name only (the archive records no direction); null = not inferred, chart draws points only","by_metric":{"% Test Accuracy":"higher","% Train Accuracy":"higher","Parameters":null,"Dev Accuracy":"higher","% Dev Accuracy":"higher","Accuracy":"higher"}},"counts":{"rows":98,"rows_with_code":73,"rows_with_paper_page":94,"rows_dated":94,"rows_using_additional_data":2},"rows":[{"rank_in_archive_order":1,"model":"UnitedSynT5 (3B)","metrics":{"% Test Accuracy":"94.7"},"uses_additional_data":true,"paper_date":"2024-12-12","paper":"/paper/first-train-to-generate-then-generate-to","paper_url":"https://arxiv.org/abs/2412.09263v2","paper_title":"First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":2,"model":"UnitedSynT5 (335M)","metrics":{"% Test Accuracy":"93.5"},"uses_additional_data":true,"paper_date":"2024-12-12","paper":"/paper/first-train-to-generate-then-generate-to","paper_url":"https://arxiv.org/abs/2412.09263v2","paper_title":"First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":3,"model":"Neural Tree Indexers for Text Understanding","metrics":{"% Test Accuracy":"93.1","Parameters":"355"},"uses_additional_data":false,"paper_date":"2021-04-29","paper":"/paper/entailment-as-few-shot-learner","paper_url":"https://arxiv.org/abs/2104.14690v1","paper_title":"Entailment as Few-Shot Learner","code":"https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/few_shot/efl","n_code_links":3,"syntology":{"n_ran":1,"n_unverified":2,"n_samples":3,"n_pointer_only_licence":0}},{"rank_in_archive_order":4,"model":"EFL (Entailment as Few-shot Learner) + RoBERTa-large","metrics":{"% Test Accuracy":"93.1","% Train Accuracy":"?","Parameters":"355m"},"uses_additional_data":false,"paper_date":"2021-04-29","paper":"/paper/entailment-as-few-shot-learner","paper_url":"https://arxiv.org/abs/2104.14690v1","paper_title":"Entailment as Few-Shot Learner","code":"https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/few_shot/efl","n_code_links":3,"syntology":{"n_ran":1,"n_unverified":2,"n_samples":3,"n_pointer_only_licence":0}},{"rank_in_archive_order":5,"model":"RoBERTa-large+Self-Explaining","metrics":{"% Test Accuracy":"92.3","Parameters":"340"},"uses_additional_data":false,"paper_date":"2020-12-03","paper":"/paper/self-explaining-structures-improve-nlp-models","paper_url":"https://arxiv.org/abs/2012.01786v2","paper_title":"Self-Explaining Structures Improve NLP Models","code":"https://github.com/ShannonAI/Self_Explaining_Structures_Improve_NLP_Models","n_code_links":1,"syntology":null},{"rank_in_archive_order":6,"model":"RoBERTa-large + self-explaining layer","metrics":{"% Test Accuracy":"92.3","% Train Accuracy":"?","Parameters":"355m+"},"uses_additional_data":false,"paper_date":"2020-12-03","paper":"/paper/self-explaining-structures-improve-nlp-models","paper_url":"https://arxiv.org/abs/2012.01786v2","paper_title":"Self-Explaining Structures Improve NLP Models","code":"https://github.com/ShannonAI/Self_Explaining_Structures_Improve_NLP_Models","n_code_links":1,"syntology":null},{"rank_in_archive_order":7,"model":"CA-MTL","metrics":{"% Test Accuracy":"92.1","% Train Accuracy":"92.6","Parameters":"340m"},"uses_additional_data":false,"paper_date":"2020-09-19","paper":"/paper/conditionally-adaptive-multi-task-learning","paper_url":"https://arxiv.org/abs/2009.09139v3","paper_title":"Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data","code":"https://github.com/CAMTL/CA-MTL","n_code_links":1,"syntology":null},{"rank_in_archive_order":8,"model":"SemBERT","metrics":{"% Test Accuracy":"91.9","% Train Accuracy":"94.4","Parameters":"339m"},"uses_additional_data":false,"paper_date":"2019-09-05","paper":"/paper/semantics-aware-bert-for-language","paper_url":"https://arxiv.org/abs/1909.02209v3","paper_title":"Semantics-aware BERT for Language Understanding","code":"https://github.com/cooelf/SemBERT","n_code_links":1,"syntology":{"n_ran":4,"n_unverified":8,"n_samples":12,"n_pointer_only_licence":0}},{"rank_in_archive_order":9,"model":"MT-DNN-SMARTLARGEv0","metrics":{"% Dev Accuracy":"92.6","% Test Accuracy":"91.7"},"uses_additional_data":false,"paper_date":"2019-11-08","paper":"/paper/smart-robust-and-efficient-fine-tuning-for","paper_url":"https://arxiv.org/abs/1911.03437v5","paper_title":"SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization","code":"https://github.com/namisan/mt-dnn","n_code_links":6,"syntology":{"n_ran":8,"n_unverified":0,"n_samples":8,"n_pointer_only_licence":1}},{"rank_in_archive_order":10,"model":"MT-DNN","metrics":{"% Test Accuracy":"91.6","% Train Accuracy":"97.2","Parameters":"330m"},"uses_additional_data":false,"paper_date":"2019-01-31","paper":"/paper/multi-task-deep-neural-networks-for-natural","paper_url":"https://arxiv.org/abs/1901.11504v2","paper_title":"Multi-Task Deep Neural Networks for Natural Language Understanding","code":"https://github.com/namisan/mt-dnn","n_code_links":7,"syntology":{"n_ran":7,"n_unverified":6,"n_samples":13,"n_pointer_only_licence":2}},{"rank_in_archive_order":11,"model":"SJRC (BERT-Large +SRL)","metrics":{"% Test Accuracy":"91.3","% Train Accuracy":"95.7","Parameters":"308m"},"uses_additional_data":false,"paper_date":"2018-09-08","paper":"/paper/i-know-what-you-want-semantic-learning-for","paper_url":"https://arxiv.org/abs/1809.02794v3","paper_title":"Explicit Contextual Semantics for Text Comprehension","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":12,"model":"Ntumpha","metrics":{"% Test Accuracy":"90.5","% Train Accuracy":"99.1","Parameters":"220"},"uses_additional_data":false,"paper_date":"2019-01-31","paper":"/paper/multi-task-deep-neural-networks-for-natural","paper_url":"https://arxiv.org/abs/1901.11504v2","paper_title":"Multi-Task Deep Neural Networks for Natural Language Understanding","code":"https://github.com/namisan/mt-dnn","n_code_links":7,"syntology":{"n_ran":7,"n_unverified":6,"n_samples":13,"n_pointer_only_licence":2}},{"rank_in_archive_order":13,"model":"Densely-Connected Recurrent and Co-Attentive Network Ensemble","metrics":{"% Test Accuracy":"90.1","% Train Accuracy":"95.0","Parameters":"53.3m"},"uses_additional_data":false,"paper_date":"2018-05-29","paper":"/paper/semantic-sentence-matching-with-densely","paper_url":"http://arxiv.org/abs/1805.11360v2","paper_title":"Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":14,"model":"MFAE","metrics":{"% Test Accuracy":"90.07","% Train Accuracy":"93.18"},"uses_additional_data":false,"paper_date":"2020-05-07","paper":"/paper/what-do-questions-exactly-ask-mfae-duplicate","paper_url":"https://epubs.siam.org/doi/10.1137/1.9781611976236.26","paper_title":"What Do Questions Exactly Ask? MFAE: Duplicate Question Identification with Multi-Fusion Asking Emphasis","code":"https://github.com/rzhangpku/MFAE","n_code_links":1,"syntology":null},{"rank_in_archive_order":15,"model":"Fine-Tuned LM-Pretrained Transformer","metrics":{"% Test Accuracy":"89.9","% Train Accuracy":"96.6","Parameters":"85m"},"uses_additional_data":false,"paper_date":"2018-06-11","paper":"/paper/improving-language-understanding-by","paper_url":"https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf","paper_title":"Improving Language Understanding by Generative Pre-Training","code":"https://github.com/huggingface/transformers","n_code_links":13,"syntology":null},{"rank_in_archive_order":16,"model":"300D DMAN Ensemble","metrics":{"% Test Accuracy":"89.6","% Train Accuracy":"96.1","Parameters":"79m"},"uses_additional_data":false,"paper_date":"2019-07-23","paper":"/paper/discourse-marker-augmented-network-with-1","paper_url":"https://arxiv.org/abs/1907.09692v1","paper_title":"Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference","code":"https://github.com/ZJULearning/DMP","n_code_links":1,"syntology":null},{"rank_in_archive_order":17,"model":"300D DMAN Ensemble","metrics":{"% Test Accuracy":"89.6","% Train Accuracy":"96.1","Parameters":"79m"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":18,"model":"150D Multiway Attention Network Ensemble","metrics":{"% Test Accuracy":"89.4","% Train Accuracy":"95.5","Parameters":"58m"},"uses_additional_data":false,"paper_date":"2018-07-01","paper":"/paper/multiway-attention-networks-for-modeling","paper_url":"https://www.ijcai.org/proceedings/2018/0613","paper_title":"Multiway Attention Networks for Modeling Sentence Pairs","code":"https://github.com/zsweet/zsw_AI_model","n_code_links":1,"syntology":null},{"rank_in_archive_order":19,"model":"450D DR-BiLSTM Ensemble","metrics":{"% Test Accuracy":"89.3","% Train Accuracy":"94.8","Parameters":"45m"},"uses_additional_data":false,"paper_date":"2018-02-15","paper":"/paper/dr-bilstm-dependent-reading-bidirectional","paper_url":"http://arxiv.org/abs/1802.05577v2","paper_title":"DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":20,"model":"300D CAFE Ensemble","metrics":{"% Test Accuracy":"89.3","% Train Accuracy":"92.5","Parameters":"17.5m"},"uses_additional_data":false,"paper_date":"2017-12-30","paper":"/paper/compare-compress-and-propagate-enhancing","paper_url":"http://arxiv.org/abs/1801.00102v2","paper_title":"Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":21,"model":"ESIM + ELMo Ensemble","metrics":{"% Test Accuracy":"89.3","% Train Accuracy":"92.1","Parameters":"40m"},"uses_additional_data":false,"paper_date":"2018-02-15","paper":"/paper/deep-contextualized-word-representations","paper_url":"http://arxiv.org/abs/1802.05365v2","paper_title":"Deep contextualized word representations","code":"https://github.com/flairNLP/flair","n_code_links":46,"syntology":{"n_ran":24,"n_unverified":34,"n_samples":58,"n_pointer_only_licence":25}},{"rank_in_archive_order":22,"model":"KIM Ensemble","metrics":{"% Test Accuracy":"89.1","% Train Accuracy":"93.6","Parameters":"43m"},"uses_additional_data":false,"paper_date":"2017-11-12","paper":"/paper/neural-natural-language-inference-models","paper_url":"http://arxiv.org/abs/1711.04289v3","paper_title":"Neural Natural Language Inference Models Enhanced with External Knowledge","code":"https://github.com/lukecq1231/kim","n_code_links":2,"syntology":null},{"rank_in_archive_order":23,"model":"SLRC","metrics":{"% Test Accuracy":"89.1","% Train Accuracy":"89.1","Parameters":"6.1m"},"uses_additional_data":false,"paper_date":"2018-09-08","paper":"/paper/i-know-what-you-want-semantic-learning-for","paper_url":"https://arxiv.org/abs/1809.02794v3","paper_title":"Explicit Contextual Semantics for Text Comprehension","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":24,"model":"RE2","metrics":{"% Test Accuracy":"88.9","% Train Accuracy":"94.0","Parameters":"2.8m"},"uses_additional_data":false,"paper_date":"2019-08-01","paper":"/paper/simple-and-effective-text-matching-with-1","paper_url":"https://arxiv.org/abs/1908.00300v1","paper_title":"Simple and Effective Text Matching with Richer Alignment Features","code":"https://github.com/alibaba-edu/simple-effective-text-matching","n_code_links":3,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":25,"model":"Densely-Connected Recurrent and Co-Attentive Network","metrics":{"% Test Accuracy":"88.9","% Train Accuracy":"93.1","Parameters":"6.7m"},"uses_additional_data":false,"paper_date":"2018-05-29","paper":"/paper/semantic-sentence-matching-with-densely","paper_url":"http://arxiv.org/abs/1805.11360v2","paper_title":"Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":26,"model":"DEIM","metrics":{"% Test Accuracy":"88.9","% Train Accuracy":"92.6","Parameters":"22m"},"uses_additional_data":false,"paper_date":"2022-03-20","paper":"/paper/deim-an-effective-deep-encoding-and","paper_url":"https://arxiv.org/abs/2203.10482v1","paper_title":"DEIM: An effective deep encoding and interaction model for sentence matching","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":27,"model":"448D Densely Interactive Inference Network (DIIN, code) Ensemble","metrics":{"% Test Accuracy":"88.9","% Train Accuracy":"92.3","Parameters":"17m"},"uses_additional_data":false,"paper_date":"2017-09-13","paper":"/paper/natural-language-inference-over-interaction-1","paper_url":"http://arxiv.org/abs/1709.04348v2","paper_title":"Natural Language Inference over Interaction Space","code":"https://github.com/YichenGong/Densely-Interactive-Inference-Network","n_code_links":2,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":28,"model":"300D DMAN","metrics":{"% Test Accuracy":"88.8","% Train Accuracy":"95.4","Parameters":"9.2m"},"uses_additional_data":false,"paper_date":"2019-07-23","paper":"/paper/discourse-marker-augmented-network-with-1","paper_url":"https://arxiv.org/abs/1907.09692v1","paper_title":"Discourse Marker Augmented Network with Reinforcement Learning for Natural Language Inference","code":"https://github.com/ZJULearning/DMP","n_code_links":1,"syntology":null},{"rank_in_archive_order":29,"model":"300D DMAN","metrics":{"% Test Accuracy":"88.8","% Train Accuracy":"95.4","Parameters":"9.2m"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":30,"model":"BiMPM Ensemble","metrics":{"% Test Accuracy":"88.8","% Train Accuracy":"93.2","Parameters":"6.4m"},"uses_additional_data":false,"paper_date":"2017-02-13","paper":"/paper/bilateral-multi-perspective-matching-for","paper_url":"http://arxiv.org/abs/1702.03814v3","paper_title":"Bilateral Multi-Perspective Matching for Natural Language Sentences","code":"https://github.com/google-research-datasets/paws","n_code_links":10,"syntology":{"n_ran":1,"n_unverified":3,"n_samples":4,"n_pointer_only_licence":1}},{"rank_in_archive_order":31,"model":"ESIM + ELMo","metrics":{"% Test Accuracy":"88.7","% Train Accuracy":"91.6","Parameters":"8.0m"},"uses_additional_data":false,"paper_date":"2018-02-15","paper":"/paper/deep-contextualized-word-representations","paper_url":"http://arxiv.org/abs/1802.05365v2","paper_title":"Deep contextualized word representations","code":"https://github.com/flairNLP/flair","n_code_links":46,"syntology":{"n_ran":24,"n_unverified":34,"n_samples":58,"n_pointer_only_licence":25}},{"rank_in_archive_order":32,"model":"KIM","metrics":{"% Test Accuracy":"88.6","% Train Accuracy":"94.1","Parameters":"4.3m"},"uses_additional_data":false,"paper_date":"2017-11-12","paper":"/paper/neural-natural-language-inference-models","paper_url":"http://arxiv.org/abs/1711.04289v3","paper_title":"Neural Natural Language Inference Models Enhanced with External Knowledge","code":"https://github.com/lukecq1231/kim","n_code_links":2,"syntology":null},{"rank_in_archive_order":33,"model":"600D ESIM + 300D Syntactic TreeLSTM","metrics":{"% Test Accuracy":"88.6","% Train Accuracy":"93.5","Parameters":"7.7m"},"uses_additional_data":false,"paper_date":"2016-09-20","paper":"/paper/enhanced-lstm-for-natural-language-inference","paper_url":"http://arxiv.org/abs/1609.06038v3","paper_title":"Enhanced LSTM for Natural Language Inference","code":"https://github.com/coetaur0/ESIM","n_code_links":12,"syntology":null},{"rank_in_archive_order":34,"model":"450D DR-BiLSTM","metrics":{"% Test Accuracy":"88.5","% Train Accuracy":"94.1","Parameters":"7.5m"},"uses_additional_data":false,"paper_date":"2018-02-15","paper":"/paper/dr-bilstm-dependent-reading-bidirectional","paper_url":"http://arxiv.org/abs/1802.05577v2","paper_title":"DR-BiLSTM: Dependent Reading Bidirectional LSTM for Natural Language Inference","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":35,"model":"Stochastic Answer Network","metrics":{"% Test Accuracy":"88.5","% Train Accuracy":"93.3","Parameters":"3.5m"},"uses_additional_data":false,"paper_date":"2018-04-21","paper":"/paper/stochastic-answer-networks-for-natural","paper_url":"http://arxiv.org/abs/1804.07888v2","paper_title":"Stochastic Answer Networks for Natural Language Inference","code":"https://github.com/kevinduh/san_mrc","n_code_links":3,"syntology":null},{"rank_in_archive_order":36,"model":"300D CAFE","metrics":{"% Test Accuracy":"88.5","% Train Accuracy":"89.8","Parameters":"4.7m"},"uses_additional_data":false,"paper_date":"2017-12-30","paper":"/paper/compare-compress-and-propagate-enhancing","paper_url":"http://arxiv.org/abs/1801.00102v2","paper_title":"Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":37,"model":"150D Multiway Attention Network","metrics":{"% Test Accuracy":"88.3","% Train Accuracy":"94.5","Parameters":"14m"},"uses_additional_data":false,"paper_date":"2018-07-01","paper":"/paper/multiway-attention-networks-for-modeling","paper_url":"https://www.ijcai.org/proceedings/2018/0613","paper_title":"Multiway Attention Networks for Modeling Sentence Pairs","code":"https://github.com/zsweet/zsw_AI_model","n_code_links":1,"syntology":null},{"rank_in_archive_order":38,"model":"Biattentive Classification Network + CoVe + Char","metrics":{"% Test Accuracy":"88.1","% Train Accuracy":"88.5","Parameters":"22m"},"uses_additional_data":false,"paper_date":"2017-08-01","paper":"/paper/learned-in-translation-contextualized-word","paper_url":"http://arxiv.org/abs/1708.00107v2","paper_title":"Learned in Translation: Contextualized Word Vectors","code":"https://github.com/salesforce/cove","n_code_links":5,"syntology":{"n_ran":3,"n_unverified":0,"n_samples":3,"n_pointer_only_licence":2}},{"rank_in_archive_order":39,"model":"aESIM","metrics":{"% Test Accuracy":"88.1"},"uses_additional_data":false,"paper_date":"2018-12-05","paper":"/paper/attention-boosted-sequential-inference-model","paper_url":"http://arxiv.org/abs/1812.01840v2","paper_title":"Attention Boosted Sequential Inference Model","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":40,"model":"448D Densely Interactive Inference Network (DIIN, code)","metrics":{"% Test Accuracy":"88.0","% Train Accuracy":"91.2","Parameters":"4.4m"},"uses_additional_data":false,"paper_date":"2017-09-13","paper":"/paper/natural-language-inference-over-interaction-1","paper_url":"http://arxiv.org/abs/1709.04348v2","paper_title":"Natural Language Inference over Interaction Space","code":"https://github.com/YichenGong/Densely-Interactive-Inference-Network","n_code_links":2,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":41,"model":"Enhanced Sequential Inference Model (Chen et al., [2017a])","metrics":{"% Test Accuracy":"88.0"},"uses_additional_data":false,"paper_date":"2016-09-20","paper":"/paper/enhanced-lstm-for-natural-language-inference","paper_url":"http://arxiv.org/abs/1609.06038v3","paper_title":"Enhanced LSTM for Natural Language Inference","code":"https://github.com/coetaur0/ESIM","n_code_links":12,"syntology":null},{"rank_in_archive_order":42,"model":"BiMPM","metrics":{"% Test Accuracy":"87.5","% Train Accuracy":"90.9","Parameters":"1.6m"},"uses_additional_data":false,"paper_date":"2017-02-13","paper":"/paper/bilateral-multi-perspective-matching-for","paper_url":"http://arxiv.org/abs/1702.03814v3","paper_title":"Bilateral Multi-Perspective Matching for Natural Language Sentences","code":"https://github.com/google-research-datasets/paws","n_code_links":10,"syntology":{"n_ran":1,"n_unverified":3,"n_samples":4,"n_pointer_only_licence":1}},{"rank_in_archive_order":43,"model":"300D re-read LSTM","metrics":{"% Test Accuracy":"87.5","% Train Accuracy":"90.7","Parameters":"2.0m"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":44,"model":"300D re-read LSTM","metrics":{"% Test Accuracy":"87.5","% Train Accuracy":"90.7","Parameters":"2.0m"},"uses_additional_data":false,"paper_date":"2016-12-01","paper":"/paper/reading-and-thinking-re-read-lstm-unit-for","paper_url":"https://aclanthology.org/C16-1270","paper_title":"Reading and Thinking: Re-read LSTM Unit for Textual Entailment Recognition","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":45,"model":"2400D Multiple-Dynamic Self-Attention Model","metrics":{"% Test Accuracy":"87.4","% Train Accuracy":"89.0","Parameters":"7.0m"},"uses_additional_data":false,"paper_date":"2018-08-22","paper":"/paper/dynamic-self-attention-computing-attention","paper_url":"http://arxiv.org/abs/1808.07383v1","paper_title":"Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding","code":"https://github.com/dsindex/iclassifier","n_code_links":1,"syntology":null},{"rank_in_archive_order":46,"model":"300D Full tree matching NTI-SLSTM-LSTM w/ global attention","metrics":{"% Test Accuracy":"87.3","% Train Accuracy":"88.5","Parameters":"3.2m"},"uses_additional_data":false,"paper_date":"2016-07-15","paper":"/paper/neural-tree-indexers-for-text-understanding","paper_url":"http://arxiv.org/abs/1607.04492v2","paper_title":"Neural Tree Indexers for Text Understanding","code":"https://bitbucket.org/tsendeemts/nti","n_code_links":1,"syntology":null},{"rank_in_archive_order":47,"model":"300D 2-layer Bi-CAS-LSTM","metrics":{"% Test Accuracy":"87"},"uses_additional_data":false,"paper_date":"2018-09-07","paper":"/paper/cell-aware-stacked-lstms-for-modeling","paper_url":"https://arxiv.org/abs/1809.02279v2","paper_title":"Cell-aware Stacked LSTMs for Modeling Sentences","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":48,"model":"200D decomposable attention feed-forward model with intra-sentence attention","metrics":{"% Test Accuracy":"86.8","% Train Accuracy":"90.5","Parameters":"580k"},"uses_additional_data":false,"paper_date":"2016-06-06","paper":"/paper/a-decomposable-attention-model-for-natural","paper_url":"http://arxiv.org/abs/1606.01933v2","paper_title":"A Decomposable Attention Model for Natural Language Inference","code":"https://github.com/dmlc/gluon-nlp","n_code_links":10,"syntology":{"n_ran":0,"n_unverified":6,"n_samples":6,"n_pointer_only_licence":0}},{"rank_in_archive_order":49,"model":"200D decomposable attention model with intra-sentence attention","metrics":{"% Test Accuracy":"86.8","% Train Accuracy":"90.5","Parameters":"580k"},"uses_additional_data":false,"paper_date":"2016-06-06","paper":"/paper/a-decomposable-attention-model-for-natural","paper_url":"http://arxiv.org/abs/1606.01933v2","paper_title":"A Decomposable Attention Model for Natural Language Inference","code":"https://github.com/dmlc/gluon-nlp","n_code_links":10,"syntology":{"n_ran":0,"n_unverified":6,"n_samples":6,"n_pointer_only_licence":0}},{"rank_in_archive_order":50,"model":"600D Dynamic Self-Attention Model","metrics":{"% Test Accuracy":"86.8","% Train Accuracy":"87.3","Parameters":"2.1m"},"uses_additional_data":false,"paper_date":"2018-08-22","paper":"/paper/dynamic-self-attention-computing-attention","paper_url":"http://arxiv.org/abs/1808.07383v1","paper_title":"Dynamic Self-Attention : Computing Attention over Words Dynamically for Sentence Embedding","code":"https://github.com/dsindex/iclassifier","n_code_links":1,"syntology":null},{"rank_in_archive_order":51,"model":"CBS-1 + ESIM","metrics":{"% Test Accuracy":"86.73"},"uses_additional_data":false,"paper_date":"2018-12-04","paper":"/paper/parameter-re-initialization-through-cyclical","paper_url":"http://arxiv.org/abs/1812.01216v1","paper_title":"Parameter Re-Initialization through Cyclical Batch Size Schedules","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":52,"model":"512D Dynamic Meta-Embeddings","metrics":{"% Test Accuracy":"86.7","% Train Accuracy":"91.6","Parameters":"9m"},"uses_additional_data":false,"paper_date":"2018-04-21","paper":"/paper/dynamic-meta-embeddings-for-improved-sentence","paper_url":"http://arxiv.org/abs/1804.07983v2","paper_title":"Dynamic Meta-Embeddings for Improved Sentence Representations","code":"https://github.com/facebookresearch/DME","n_code_links":3,"syntology":null},{"rank_in_archive_order":53,"model":"600D BiLSTM with generalized pooling","metrics":{"% Test Accuracy":"86.6","% Train Accuracy":"94.9","Parameters":"65m"},"uses_additional_data":false,"paper_date":"2018-06-26","paper":"/paper/enhancing-sentence-embedding-with-generalized","paper_url":"http://arxiv.org/abs/1806.09828v1","paper_title":"Enhancing Sentence Embedding with Generalized Pooling","code":"https://github.com/lukecq1231/generalized-pooling","n_code_links":1,"syntology":null},{"rank_in_archive_order":54,"model":"600D Hierarchical BiLSTM with Max Pooling (HBMP, code)","metrics":{"% Test Accuracy":"86.6","% Train Accuracy":"89.9","Parameters":"22m"},"uses_additional_data":false,"paper_date":"2018-08-27","paper":"/paper/natural-language-inference-with-hierarchical","paper_url":"https://arxiv.org/abs/1808.08762v2","paper_title":"Sentence Embeddings in NLI with Iterative Refinement Encoders","code":"https://github.com/Helsinki-NLP/HBMP","n_code_links":1,"syntology":null},{"rank_in_archive_order":55,"model":"Densely-Connected Recurrent and Co-Attentive Network (encoder)","metrics":{"% Test Accuracy":"86.5","% Train Accuracy":"91.4","Parameters":"5.6m"},"uses_additional_data":false,"paper_date":"2018-05-29","paper":"/paper/semantic-sentence-matching-with-densely","paper_url":"http://arxiv.org/abs/1805.11360v2","paper_title":"Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":56,"model":"300D Reinforced Self-Attention Network","metrics":{"% Test Accuracy":"86.3","% Train Accuracy":"92.6","Parameters":"3.1m"},"uses_additional_data":false,"paper_date":"2018-01-31","paper":"/paper/reinforced-self-attention-network-a-hybrid-of","paper_url":"http://arxiv.org/abs/1801.10296v2","paper_title":"Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling","code":"https://github.com/taoshen58/DiSAN","n_code_links":1,"syntology":null},{"rank_in_archive_order":57,"model":"Distance-based Self-Attention Network","metrics":{"% Test Accuracy":"86.3","% Train Accuracy":"89.6","Parameters":"4.7m"},"uses_additional_data":false,"paper_date":"2017-12-06","paper":"/paper/distance-based-self-attention-network-for","paper_url":"http://arxiv.org/abs/1712.02047v1","paper_title":"Distance-based Self-Attention Network for Natural Language Inference","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":58,"model":"200D decomposable attention feed-forward model","metrics":{"% Test Accuracy":"86.3","% Train Accuracy":"89.5","Parameters":"380k"},"uses_additional_data":false,"paper_date":"2016-06-06","paper":"/paper/a-decomposable-attention-model-for-natural","paper_url":"http://arxiv.org/abs/1606.01933v2","paper_title":"A Decomposable Attention Model for Natural Language Inference","code":"https://github.com/dmlc/gluon-nlp","n_code_links":10,"syntology":{"n_ran":0,"n_unverified":6,"n_samples":6,"n_pointer_only_licence":0}},{"rank_in_archive_order":59,"model":"200D decomposable attention model","metrics":{"% Test Accuracy":"86.3","% Train Accuracy":"89.5","Parameters":"380k"},"uses_additional_data":false,"paper_date":"2016-06-06","paper":"/paper/a-decomposable-attention-model-for-natural","paper_url":"http://arxiv.org/abs/1606.01933v2","paper_title":"A Decomposable Attention Model for Natural Language Inference","code":"https://github.com/dmlc/gluon-nlp","n_code_links":10,"syntology":{"n_ran":0,"n_unverified":6,"n_samples":6,"n_pointer_only_licence":0}},{"rank_in_archive_order":60,"model":"450D LSTMN with deep attention fusion","metrics":{"% Test Accuracy":"86.3","% Train Accuracy":"88.5","Parameters":"3.4m"},"uses_additional_data":false,"paper_date":"2016-01-25","paper":"/paper/long-short-term-memory-networks-for-machine","paper_url":"http://arxiv.org/abs/1601.06733v7","paper_title":"Long Short-Term Memory-Networks for Machine Reading","code":"https://github.com/JRC1995/Abstractive-Summarization","n_code_links":3,"syntology":{"n_ran":2,"n_unverified":0,"n_samples":2,"n_pointer_only_licence":2}},{"rank_in_archive_order":61,"model":"300D mLSTM word-by-word attention model","metrics":{"% Test Accuracy":"86.1","% Train Accuracy":"92.0","Parameters":"1.9m"},"uses_additional_data":false,"paper_date":"2015-12-30","paper":"/paper/learning-natural-language-inference-with-lstm","paper_url":"http://arxiv.org/abs/1512.08849v2","paper_title":"Learning Natural Language Inference with LSTM","code":"https://github.com/shuohangwang/SeqMatchSeq","n_code_links":4,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":1}},{"rank_in_archive_order":62,"model":"600D Gumbel TreeLSTM encoders","metrics":{"% Test Accuracy":"86.0","% Train Accuracy":"93.1","Parameters":"10m"},"uses_additional_data":false,"paper_date":"2017-07-10","paper":"/paper/learning-to-compose-task-specific-tree","paper_url":"http://arxiv.org/abs/1707.02786v4","paper_title":"Learning to Compose Task-Specific Tree Structures","code":"https://github.com/jihunchoi/unsupervised-treelstm","n_code_links":1,"syntology":null},{"rank_in_archive_order":63,"model":"600D Residual stacked encoders","metrics":{"% Test Accuracy":"86.0","% Train Accuracy":"91.0","Parameters":"29m"},"uses_additional_data":false,"paper_date":"2017-08-07","paper":"/paper/shortcut-stacked-sentence-encoders-for-multi","paper_url":"http://arxiv.org/abs/1708.02312v2","paper_title":"Shortcut-Stacked Sentence Encoders for Multi-Domain Inference","code":"https://github.com/easonnie/multiNLI_encoder","n_code_links":3,"syntology":null},{"rank_in_archive_order":64,"model":"Star-Transformer (no cross sentence attention)","metrics":{"% Test Accuracy":"86.0"},"uses_additional_data":false,"paper_date":"2019-02-25","paper":"/paper/star-transformer","paper_url":"https://arxiv.org/abs/1902.09113v3","paper_title":"Star-Transformer","code":"https://github.com/dmlc/dgl","n_code_links":2,"syntology":{"n_ran":0,"n_unverified":1,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":65,"model":"300D CAFE (no cross-sentence attention)","metrics":{"% Test Accuracy":"85.9","% Train Accuracy":"87.3","Parameters":"3.7m"},"uses_additional_data":false,"paper_date":"2017-12-30","paper":"/paper/compare-compress-and-propagate-enhancing","paper_url":"http://arxiv.org/abs/1801.00102v2","paper_title":"Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":66,"model":"1200D REGMAPR (Base+Reg)","metrics":{"% Test Accuracy":"85.9","% Train Accuracy":"–","Parameters":"–"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":67,"model":"300D Residual stacked encoders","metrics":{"% Test Accuracy":"85.7","% Train Accuracy":"89.8","Parameters":"9.7m"},"uses_additional_data":false,"paper_date":"2017-08-07","paper":"/paper/shortcut-stacked-sentence-encoders-for-multi","paper_url":"http://arxiv.org/abs/1708.02312v2","paper_title":"Shortcut-Stacked Sentence Encoders for Multi-Domain Inference","code":"https://github.com/easonnie/multiNLI_encoder","n_code_links":3,"syntology":null},{"rank_in_archive_order":68,"model":"300D LSTMN with deep attention fusion","metrics":{"% Test Accuracy":"85.7","% Train Accuracy":"87.3","Parameters":"1.7m"},"uses_additional_data":false,"paper_date":"2016-01-25","paper":"/paper/long-short-term-memory-networks-for-machine","paper_url":"http://arxiv.org/abs/1601.06733v7","paper_title":"Long Short-Term Memory-Networks for Machine Reading","code":"https://github.com/JRC1995/Abstractive-Summarization","n_code_links":3,"syntology":{"n_ran":2,"n_unverified":0,"n_samples":2,"n_pointer_only_licence":2}},{"rank_in_archive_order":69,"model":"300D Gumbel TreeLSTM encoders","metrics":{"% Test Accuracy":"85.6","% Train Accuracy":"91.2","Parameters":"2.9m"},"uses_additional_data":false,"paper_date":"2017-07-10","paper":"/paper/learning-to-compose-task-specific-tree","paper_url":"http://arxiv.org/abs/1707.02786v4","paper_title":"Learning to Compose Task-Specific Tree Structures","code":"https://github.com/jihunchoi/unsupervised-treelstm","n_code_links":1,"syntology":null},{"rank_in_archive_order":70,"model":"300D Directional self-attention network encoders","metrics":{"% Test Accuracy":"85.6","% Train Accuracy":"91.1","Parameters":"2.4m"},"uses_additional_data":false,"paper_date":"2017-09-14","paper":"/paper/disan-directional-self-attention-network-for","paper_url":"http://arxiv.org/abs/1709.04696v3","paper_title":"DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding","code":"https://github.com/taoshen58/DiSAN","n_code_links":3,"syntology":null},{"rank_in_archive_order":71,"model":"600D (300+300) Deep Gated Attn. BiLSTM encoders","metrics":{"% Test Accuracy":"85.5","% Train Accuracy":"90.5","Parameters":"12m"},"uses_additional_data":false,"paper_date":"2017-08-04","paper":"/paper/recurrent-neural-network-based-sentence","paper_url":"http://arxiv.org/abs/1708.01353v1","paper_title":"Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference","code":"https://github.com/lukecq1231/enc_nli","n_code_links":2,"syntology":null},{"rank_in_archive_order":72,"model":"300D MMA-NSE encoders with attention","metrics":{"% Test Accuracy":"85.4","% Train Accuracy":"86.9","Parameters":"3.2m"},"uses_additional_data":false,"paper_date":"2016-07-14","paper":"/paper/neural-semantic-encoders","paper_url":"http://arxiv.org/abs/1607.04315v3","paper_title":"Neural Semantic Encoders","code":"https://github.com/Smerity/keras_snli","n_code_links":3,"syntology":null},{"rank_in_archive_order":73,"model":"50D stacked TC-LSTMs","metrics":{"% Test Accuracy":"85.1","% Train Accuracy":"86.7","Parameters":"190k"},"uses_additional_data":false,"paper_date":"2016-05-18","paper":"/paper/modelling-interaction-of-sentence-pair-with","paper_url":"http://arxiv.org/abs/1605.05573v2","paper_title":"Modelling Interaction of Sentence Pair with coupled-LSTMs","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":74,"model":"600D (300+300) BiLSTM encoders with intra-attention and symbolic preproc.","metrics":{"% Test Accuracy":"85.0","% Train Accuracy":"85.9","Parameters":"2.8m"},"uses_additional_data":false,"paper_date":"2016-05-30","paper":"/paper/learning-natural-language-inference-using","paper_url":"http://arxiv.org/abs/1605.09090v1","paper_title":"Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention","code":"https://github.com/Smerity/keras_snli","n_code_links":2,"syntology":null},{"rank_in_archive_order":75,"model":"Stacked Bi-LSTMs (shortcut connections, max-pooling)","metrics":{"% Test Accuracy":"84.8"},"uses_additional_data":false,"paper_date":"2018-11-02","paper":"/paper/combining-similarity-features-and-deep","paper_url":"http://arxiv.org/abs/1811.00706v1","paper_title":"Combining Similarity Features and Deep Representation Learning for Stance Detection in the Context of Checking Fake News","code":"https://github.com/imran3180/pytorch-nli","n_code_links":3,"syntology":null},{"rank_in_archive_order":76,"model":"300D NSE encoders","metrics":{"% Test Accuracy":"84.6","% Train Accuracy":"86.2","Parameters":"3.0m"},"uses_additional_data":false,"paper_date":"2016-07-14","paper":"/paper/neural-semantic-encoders","paper_url":"http://arxiv.org/abs/1607.04315v3","paper_title":"Neural Semantic Encoders","code":"https://github.com/Smerity/keras_snli","n_code_links":3,"syntology":null},{"rank_in_archive_order":77,"model":"100D DF-LSTM","metrics":{"% Test Accuracy":"84.6","% Train Accuracy":"85.2","Parameters":"320k"},"uses_additional_data":false,"paper_date":"2016-08-01","paper":"/paper/deep-fusion-lstms-for-text-semantic-matching","paper_url":"https://aclanthology.org/P16-1098","paper_title":"Deep Fusion LSTMs for Text Semantic Matching","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":78,"model":"4096D BiLSTM with max-pooling","metrics":{"% Test Accuracy":"84.5","% Train Accuracy":"85.6","Parameters":"40m"},"uses_additional_data":false,"paper_date":"2017-05-05","paper":"/paper/supervised-learning-of-universal-sentence","paper_url":"http://arxiv.org/abs/1705.02364v5","paper_title":"Supervised Learning of Universal Sentence Representations from Natural Language Inference Data","code":"https://github.com/facebookresearch/InferSent","n_code_links":23,"syntology":{"n_ran":6,"n_unverified":1,"n_samples":7,"n_pointer_only_licence":7}},{"rank_in_archive_order":79,"model":"Bi-LSTM sentence encoder (max-pooling)","metrics":{"% Test Accuracy":"84.5"},"uses_additional_data":false,"paper_date":"2018-11-02","paper":"/paper/combining-similarity-features-and-deep","paper_url":"http://arxiv.org/abs/1811.00706v1","paper_title":"Combining Similarity Features and Deep Representation Learning for Stance Detection in the Context of Checking Fake News","code":"https://github.com/imran3180/pytorch-nli","n_code_links":3,"syntology":null},{"rank_in_archive_order":80,"model":"Stacked Bi-LSTMs (shortcut connections, max-pooling, attention)","metrics":{"% Test Accuracy":"84.4"},"uses_additional_data":false,"paper_date":"2018-11-02","paper":"/paper/combining-similarity-features-and-deep","paper_url":"http://arxiv.org/abs/1811.00706v1","paper_title":"Combining Similarity Features and Deep Representation Learning for Stance Detection in the Context of Checking Fake News","code":"https://github.com/imran3180/pytorch-nli","n_code_links":3,"syntology":null},{"rank_in_archive_order":81,"model":"600D (300+300) BiLSTM encoders with intra-attention","metrics":{"% Test Accuracy":"84.2","% Train Accuracy":"84.5","Parameters":"2.8m"},"uses_additional_data":false,"paper_date":"2016-05-30","paper":"/paper/learning-natural-language-inference-using","paper_url":"http://arxiv.org/abs/1605.09090v1","paper_title":"Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention","code":"https://github.com/Smerity/keras_snli","n_code_links":2,"syntology":null},{"rank_in_archive_order":82,"model":"SWEM-max","metrics":{"% Test Accuracy":"83.8"},"uses_additional_data":false,"paper_date":"2018-05-24","paper":"/paper/baseline-needs-more-love-on-simple-word","paper_url":"http://arxiv.org/abs/1805.09843v1","paper_title":"Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms","code":"https://github.com/dinghanshen/SWEM","n_code_links":2,"syntology":null},{"rank_in_archive_order":83,"model":"100D LSTMs w/ word-by-word attention","metrics":{"% Test Accuracy":"83.5","% Train Accuracy":"85.3","Parameters":"250k"},"uses_additional_data":false,"paper_date":"2015-09-22","paper":"/paper/reasoning-about-entailment-with-neural","paper_url":"http://arxiv.org/abs/1509.06664v4","paper_title":"Reasoning about Entailment with Neural Attention","code":"https://github.com/shyamupa/snli-entailment","n_code_links":7,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":1}},{"rank_in_archive_order":84,"model":"300D NTI-SLSTM-LSTM encoders","metrics":{"% Test Accuracy":"83.4","% Train Accuracy":"82.5","Parameters":"4.0m"},"uses_additional_data":false,"paper_date":"2016-07-15","paper":"/paper/neural-tree-indexers-for-text-understanding","paper_url":"http://arxiv.org/abs/1607.04492v2","paper_title":"Neural Tree Indexers for Text Understanding","code":"https://bitbucket.org/tsendeemts/nti","n_code_links":1,"syntology":null},{"rank_in_archive_order":85,"model":"600D (300+300) BiLSTM encoders","metrics":{"% Test Accuracy":"83.3","% Train Accuracy":"86.4","Parameters":"2.0m"},"uses_additional_data":false,"paper_date":"2016-05-30","paper":"/paper/learning-natural-language-inference-using","paper_url":"http://arxiv.org/abs/1605.09090v1","paper_title":"Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention","code":"https://github.com/Smerity/keras_snli","n_code_links":2,"syntology":null},{"rank_in_archive_order":86,"model":"300D SPINN-PI encoders","metrics":{"% Test Accuracy":"83.2","% Train Accuracy":"89.2","Parameters":"3.7m"},"uses_additional_data":false,"paper_date":"2016-03-19","paper":"/paper/a-fast-unified-model-for-parsing-and-sentence","paper_url":"http://arxiv.org/abs/1603.06021v3","paper_title":"A Fast Unified Model for Parsing and Sentence Understanding","code":"https://github.com/stanfordnlp/spinn","n_code_links":3,"syntology":{"n_ran":0,"n_unverified":6,"n_samples":6,"n_pointer_only_licence":0}},{"rank_in_archive_order":87,"model":"300D Tree-based CNN encoders","metrics":{"% Test Accuracy":"82.1","% Train Accuracy":"83.3","Parameters":"3.5m"},"uses_additional_data":false,"paper_date":"2015-12-28","paper":"/paper/natural-language-inference-by-tree-based","paper_url":"http://arxiv.org/abs/1512.08422v3","paper_title":"Natural Language Inference by Tree-Based Convolution and Heuristic Matching","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":88,"model":"1024D GRU encoders w/ unsupervised 'skip-thoughts' pre-training","metrics":{"% Test Accuracy":"81.4","% Train Accuracy":"98.8","Parameters":"15m"},"uses_additional_data":false,"paper_date":"2015-11-19","paper":"/paper/order-embeddings-of-images-and-language","paper_url":"http://arxiv.org/abs/1511.06361v6","paper_title":"Order-Embeddings of Images and Language","code":"https://github.com/ivendrov/order-embedding","n_code_links":2,"syntology":null},{"rank_in_archive_order":89,"model":"DELTA (LSTM)","metrics":{"% Test Accuracy":"80.7"},"uses_additional_data":false,"paper_date":"2019-08-02","paper":"/paper/delta-a-deep-learning-based-language","paper_url":"https://arxiv.org/abs/1908.01853v1","paper_title":"DELTA: A DEep learning based Language Technology plAtform","code":"https://github.com/didi/delta","n_code_links":2,"syntology":null},{"rank_in_archive_order":90,"model":"300D LSTM encoders","metrics":{"% Test Accuracy":"80.6","% Train Accuracy":"83.9","Parameters":"3.0m"},"uses_additional_data":false,"paper_date":"2016-03-19","paper":"/paper/a-fast-unified-model-for-parsing-and-sentence","paper_url":"http://arxiv.org/abs/1603.06021v3","paper_title":"A Fast Unified Model for Parsing and Sentence Understanding","code":"https://github.com/stanfordnlp/spinn","n_code_links":3,"syntology":{"n_ran":0,"n_unverified":6,"n_samples":6,"n_pointer_only_licence":0}},{"rank_in_archive_order":91,"model":"+ Unigram and bigram features","metrics":{"% Test Accuracy":"78.2","% Train Accuracy":"99.7"},"uses_additional_data":false,"paper_date":"2015-08-21","paper":"/paper/a-large-annotated-corpus-for-learning-natural","paper_url":"http://arxiv.org/abs/1508.05326v1","paper_title":"A large annotated corpus for learning natural language inference","code":"https://github.com/kawine/dataset_difficulty","n_code_links":3,"syntology":{"n_ran":3,"n_unverified":0,"n_samples":3,"n_pointer_only_licence":3}},{"rank_in_archive_order":92,"model":"100D LSTM encoders","metrics":{"% Test Accuracy":"77.6","% Train Accuracy":"84.8","Parameters":"220k"},"uses_additional_data":false,"paper_date":"2015-08-21","paper":"/paper/a-large-annotated-corpus-for-learning-natural","paper_url":"http://arxiv.org/abs/1508.05326v1","paper_title":"A large annotated corpus for learning natural language inference","code":"https://github.com/kawine/dataset_difficulty","n_code_links":3,"syntology":{"n_ran":3,"n_unverified":0,"n_samples":3,"n_pointer_only_licence":3}},{"rank_in_archive_order":93,"model":"Unlexicalized features","metrics":{"% Test Accuracy":"50.4","% Train Accuracy":"49.4"},"uses_additional_data":false,"paper_date":"2015-08-21","paper":"/paper/a-large-annotated-corpus-for-learning-natural","paper_url":"http://arxiv.org/abs/1508.05326v1","paper_title":"A large annotated corpus for learning natural language inference","code":"https://github.com/kawine/dataset_difficulty","n_code_links":3,"syntology":{"n_ran":3,"n_unverified":0,"n_samples":3,"n_pointer_only_licence":3}},{"rank_in_archive_order":94,"model":"MT-DNN-SMART_100%ofTrainingData","metrics":{"Dev Accuracy":"91.6"},"uses_additional_data":false,"paper_date":"2019-11-08","paper":"/paper/smart-robust-and-efficient-fine-tuning-for","paper_url":"https://arxiv.org/abs/1911.03437v5","paper_title":"SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization","code":"https://github.com/namisan/mt-dnn","n_code_links":6,"syntology":{"n_ran":8,"n_unverified":0,"n_samples":8,"n_pointer_only_licence":1}},{"rank_in_archive_order":95,"model":"MT-DNN-SMART_10%ofTrainingData","metrics":{"Dev Accuracy":"88.7"},"uses_additional_data":false,"paper_date":"2019-11-08","paper":"/paper/smart-robust-and-efficient-fine-tuning-for","paper_url":"https://arxiv.org/abs/1911.03437v5","paper_title":"SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization","code":"https://github.com/namisan/mt-dnn","n_code_links":6,"syntology":{"n_ran":8,"n_unverified":0,"n_samples":8,"n_pointer_only_licence":1}},{"rank_in_archive_order":96,"model":"MT-DNN-SMART_1%ofTrainingData","metrics":{"Dev Accuracy":"86"},"uses_additional_data":false,"paper_date":"2019-11-08","paper":"/paper/smart-robust-and-efficient-fine-tuning-for","paper_url":"https://arxiv.org/abs/1911.03437v5","paper_title":"SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization","code":"https://github.com/namisan/mt-dnn","n_code_links":6,"syntology":{"n_ran":8,"n_unverified":0,"n_samples":8,"n_pointer_only_licence":1}},{"rank_in_archive_order":97,"model":"MT-DNN-SMART_0.1%ofTrainingData","metrics":{"Dev Accuracy":"82.7"},"uses_additional_data":false,"paper_date":"2019-11-08","paper":"/paper/smart-robust-and-efficient-fine-tuning-for","paper_url":"https://arxiv.org/abs/1911.03437v5","paper_title":"SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization","code":"https://github.com/namisan/mt-dnn","n_code_links":6,"syntology":{"n_ran":8,"n_unverified":0,"n_samples":8,"n_pointer_only_licence":1}},{"rank_in_archive_order":98,"model":"SplitEE-S","metrics":{"Accuracy":"79.0"},"uses_additional_data":false,"paper_date":"2023-09-17","paper":"/paper/splitee-early-exit-in-deep-neural-networks","paper_url":"https://arxiv.org/abs/2309.09195v1","paper_title":"SplitEE: Early Exit in Deep Neural Networks with Split Computing","code":"https://github.com/Div290/SplitEE/blob/main/README.md","n_code_links":1,"syntology":null}],"since_archive":{"claim":"Results that newer papers report for their own method, placed here by Syntology. A model pointed at the cell in the paper's own table; the number was read from that cell and checked against this leaderboard's metric, dataset, split and scale; an independent check that saw this leaderboard's other rows and every other leaderboard on the same dataset accepted it. Not reviewed by the paper's authors or by the archive's editors, and not ranked against the archive rows.","extraction_file_present":true,"measurement":{"test_papers":883,"papers_with_output":881,"judged_true":108,"judged":110,"wilson95_lower":0.9361,"measured_on":"2026-09-24","frozen_commit":"0e3de0df94"},"measurement_note":"blind adjudication of accepted entries on a held-out split of archive papers, rules frozen before the test","coverage":{"sentence":"Syntology has checked 7,081 of the 9,623 papers on this site that are newer than the archive; results from the others appear after they are checked.","complete":false,"papers_newer_than_archive":9623,"papers_checked":7081,"papers_extracted_not_yet_verified":217,"boards_without_verdict":27,"papers_not_yet_extracted":2325},"order":"newest first by month (arXiv date, else the arXiv-id month), then arXiv id descending","columns":[],"entries":[]},"syntology":{"read_at":"2026-09-25T09:33:49+00:00","claim":"Per row: N of M harvested code samples from that row's paper executed on a synthesized fixture; the other M-N are unverified. Not a reproduction of the row's number; not a correctness claim. n_pointer_only_licence counts samples the site points at rather than redistributes (a licence axis, independent of ran/unverified).","rows_with_graph_line":33,"rows_with_any_sample_ran":26,"distinct_papers_with_graph_line":17,"distinct_papers_with_any_sample_ran":14,"samples_over_distinct_papers":{"n_ran":63,"n_unverified":67,"n_samples":130,"n_pointer_only_licence":45,"note":"each paper (arXiv id) counted once, however many rows it is behind; this is the page-level figure"},"samples_row_weighted":{"n_ran":137,"n_unverified":136,"n_samples":273,"n_pointer_only_licence":85,"note":"row-weighted: a paper behind several rows is counted once per row; inflated relative to samples_over_distinct_papers by design, kept for readers summing the per-row syntology blocks"}}}