{"url":"/sota/question-answering-on-squad20","task":{"name":"Question Answering","url":"/task/question-answering","note":null},"dataset":{"name":"SQuAD2.0","url":"/dataset/squad"},"category":"Natural Language Processing","categories":["Miscellaneous","Natural Language Processing","Reasoning"],"category_note":null,"description":"Question answering can be segmented into domain-specific tasks like community question answering and knowledge-base question answering. Popular benchmark datasets for evaluation question answering systems include [SQuAD](/dataset/squad), [HotPotQA](/dataset/hotpotqa), [bAbI](/dataset/babi-1), [TriviaQA](/dataset/triviaqa), [WikiQA](/dataset/wikiqa), and many others. Models for question answering are typically evaluated on metrics like EM and F1. Some recent top performing models are T5 and XLNet.\r\n\r\n( Image credit: [SQuAD](https://rajpurkar.github.io/mlx/qa-and-squad/) )","description_from":"task","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","rank":"the archive's row order at snapshot; not re-ranked","rows_end_at":"2025-07-28","rows_withheld_as_spam":0,"metric_values":"the archive's strings, untouched"},"metrics":["EM","F1"],"metric_direction":{"note":"inferred from the metric name only (the archive records no direction); null = not inferred, chart draws points only","by_metric":{"EM":null,"F1":"higher"}},"counts":{"rows":286,"rows_with_code":26,"rows_with_paper_page":27,"rows_dated":27,"rows_using_additional_data":0},"rows":[{"rank_in_archive_order":1,"model":"IE-Net (ensemble)","metrics":{"EM":"90.939","F1":"93.214"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":2,"model":"FPNet  (ensemble)","metrics":{"EM":"90.871","F1":"93.183"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":3,"model":"IE-NetV2 (ensemble)","metrics":{"EM":"90.860","F1":"93.100"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":4,"model":"SA-Net on Albert (ensemble)","metrics":{"EM":"90.724","F1":"93.011"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":5,"model":"SA-Net-V2 (ensemble)","metrics":{"EM":"90.679","F1":"92.948"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":6,"model":"FPNet (ensemble)","metrics":{"EM":"90.600","F1":"92.899"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":7,"model":"Retro-Reader (ensemble)","metrics":{"EM":"90.578","F1":"92.978"},"uses_additional_data":false,"paper_date":"2020-01-27","paper":"/paper/retrospective-reader-for-machine-reading","paper_url":"https://arxiv.org/abs/2001.09694v4","paper_title":"Retrospective Reader for Machine Reading Comprehension","code":"https://github.com/cooelf/AwesomeMRC","n_code_links":2,"syntology":null},{"rank_in_archive_order":8,"model":"EntitySpanFocusV2 (ensemble)","metrics":{"EM":"90.521","F1":"92.824"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":9,"model":"TransNets + SFVerifier + SFEnsembler (ensemble)","metrics":{"EM":"90.487","F1":"92.894"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":10,"model":"EntitySpanFocus+AT (ensemble)","metrics":{"EM":"90.454","F1":"92.748"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":11,"model":"ATRLP+PV (ensemble)","metrics":{"EM":"90.442","F1":"92.877"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":12,"model":"LANetV2 (ensemble)","metrics":{"EM":"90.420","F1":"92.807"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":13,"model":"ELECTRA+ALBERT+EntitySpanFocus (ensemble)","metrics":{"EM":"90.420","F1":"92.799"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":14,"model":"ALBERT + DAAF + Verifier","metrics":{"EM":"90.386","F1":"92.777"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":15,"model":"electra+nlayers+adv+ds","metrics":{"EM":"90.284","F1":"92.691"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":16,"model":"MixEnsemble (ensemble)","metrics":{"EM":"90.194","F1":"92.594"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":17,"model":"electra+nlayers (ensemble)","metrics":{"EM":"90.126","F1":"92.535"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":18,"model":"Retro-Reader on ALBERT (ensemble)","metrics":{"EM":"90.115","F1":"92.580"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":19,"model":"Retro-Reader on ALBERT (ensemble)","metrics":{"EM":"90.115","F1":"92.580"},"uses_additional_data":false,"paper_date":"2020-01-27","paper":"/paper/retrospective-reader-for-machine-reading","paper_url":"https://arxiv.org/abs/2001.09694v4","paper_title":"Retrospective Reader for Machine Reading Comprehension","code":"https://github.com/cooelf/AwesomeMRC","n_code_links":2,"syntology":null},{"rank_in_archive_order":20,"model":"ANet","metrics":{"EM":"90.081","F1":"92.457"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":21,"model":"Answer Dependent Classify (single model)","metrics":{"EM":"90.059","F1":"92.517"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":22,"model":"electra+nlayers+kdav (ensemble)","metrics":{"EM":"90.002","F1":"92.497"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":23,"model":"ALBERT + DAAF + Verifier (ensemble)","metrics":{"EM":"90.002","F1":"92.425"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":24,"model":"LANet (ensemble)","metrics":{"EM":"89.923","F1":"92.425"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":25,"model":"Electra-nlayers (ensemble)","metrics":{"EM":"89.777","F1":"92.312"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":26,"model":"Albert_Verifier_AA_Net (ensemble)","metrics":{"EM":"89.743","F1":"92.180"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":27,"model":"ALBERT (ensemble model)","metrics":{"EM":"89.731","F1":"92.215"},"uses_additional_data":false,"paper_date":"2019-09-26","paper":"/paper/albert-a-lite-bert-for-self-supervised","paper_url":"https://arxiv.org/abs/1909.11942v6","paper_title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations","code":"https://github.com/huggingface/transformers","n_code_links":48,"syntology":{"n_ran":46,"n_unverified":80,"n_samples":126,"n_pointer_only_licence":22}},{"rank_in_archive_order":28,"model":"Span Extract + Classify (single model)","metrics":{"EM":"89.562","F1":"92.226"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":29,"model":"Retro-Reader on ELECTRA (single model)","metrics":{"EM":"89.562","F1":"92.052"},"uses_additional_data":false,"paper_date":"2020-01-27","paper":"/paper/retrospective-reader-for-machine-reading","paper_url":"https://arxiv.org/abs/2001.09694v4","paper_title":"Retrospective Reader for Machine Reading Comprehension","code":"https://github.com/cooelf/AwesomeMRC","n_code_links":2,"syntology":null},{"rank_in_archive_order":30,"model":"ELECTRA+ATRLP+PV (single model)","metrics":{"EM":"89.551","F1":"92.366"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":31,"model":"albert+transform+verify (ensemble)","metrics":{"EM":"89.528","F1":"92.059"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":32,"model":"albert+KD+transfer (ensemble)","metrics":{"EM":"89.461","F1":"92.134"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":33,"model":"ROaD-Electra (single model)","metrics":{"EM":"89.449","F1":"92.118"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":34,"model":"ROaD-Electra","metrics":{"EM":"89.449","F1":"92.118"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":35,"model":"Albert-nlayers (ensemble)","metrics":{"EM":"89.404","F1":"91.964"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":36,"model":"ELECTRA + E-Verifier (ensemble)","metrics":{"EM":"89.348","F1":"91.985"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":37,"model":"ELECTRA + ROBERTA + ALBERT (ensemble)","metrics":{"EM":"89.325","F1":"91.994"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":38,"model":"2task (single model)","metrics":{"EM":"89.325","F1":"91.939"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":39,"model":"Deberta","metrics":{"EM":"89.235","F1":"91.900"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":40,"model":"ALBERT + MTDA + SFVerifier (ensemble model)","metrics":{"EM":"89.235","F1":"91.739"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":41,"model":"ALBert-LSTM (ensemble)","metrics":{"EM":"89.224","F1":"91.853"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":42,"model":"ALBERT + SFVerifier (ensemble model)","metrics":{"EM":"89.133","F1":"91.666"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":43,"model":"ELECTRA+RL+EV (single model)","metrics":{"EM":"89.021","F1":"91.765"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":44,"model":"electra & albert (ensemble)","metrics":{"EM":"88.998","F1":"91.635"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":45,"model":"AE-TEST","metrics":{"EM":"88.998","F1":"91.635"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":46,"model":"ELECTRA+EntitySpanFocus (Single model)","metrics":{"EM":"88.874","F1":"91.546"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":47,"model":"SA-Net on Electra (single model)","metrics":{"EM":"88.851","F1":"91.486"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":48,"model":"ALBERT+Entailment DA (ensemble)","metrics":{"EM":"88.761","F1":"91.745"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":49,"model":"ELECTRA (single model)","metrics":{"EM":"88.716","F1":"91.365"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":50,"model":"Tuned ALBERT (ensemble model)","metrics":{"EM":"88.637","F1":"91.230"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":51,"model":"ELECTRA_ATT (single model)","metrics":{"EM":"88.614","F1":"91.303"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":52,"model":"Deberta+prefix","metrics":{"EM":"88.603","F1":"91.299"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":53,"model":"ALBERT (Single model)","metrics":{"EM":"88.592","F1":"91.286"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":54,"model":"XLNet + DAAF + Verifier (ensemble)","metrics":{"EM":"88.592","F1":"90.859"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":55,"model":"ALBERT + IG + NE (single model)","metrics":{"EM":"88.569","F1":"91.287"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":56,"model":"ALBERT + IG (single model)","metrics":{"EM":"88.524","F1":"91.256"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":57,"model":"aanet_v2.0 (single model)","metrics":{"EM":"88.434","F1":"90.918"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":58,"model":"albert+verifier (single model)","metrics":{"EM":"88.355","F1":"91.019"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":59,"model":"albert+KD+transfer (single)","metrics":{"EM":"88.298","F1":"91.078"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":60,"model":"UPM (ensemble)","metrics":{"EM":"88.231","F1":"90.713"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":61,"model":"ALBERT + SFVerifier (single model)","metrics":{"EM":"88.197","F1":"90.830"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":62,"model":"{alber_m_transfor} (single model)","metrics":{"EM":"88.186","F1":"90.939"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":63,"model":"XLNet + SG-Net Verifier (ensemble)","metrics":{"EM":"88.174","F1":"90.702"},"uses_additional_data":false,"paper_date":"2019-08-14","paper":"/paper/sg-net-syntax-guided-machine-reading","paper_url":"https://arxiv.org/abs/1908.05147v3","paper_title":"SG-Net: Syntax-Guided Machine Reading Comprehension","code":"https://github.com/cooelf/SG-Net","n_code_links":1,"syntology":null},{"rank_in_archive_order":64,"model":"Retro-Reader on ALBERT (single model)","metrics":{"EM":"88.107","F1":"91.419"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":65,"model":"Retro-Reader on ALBERT (single model)","metrics":{"EM":"88.107","F1":"91.419"},"uses_additional_data":false,"paper_date":"2020-01-27","paper":"/paper/retrospective-reader-for-machine-reading","paper_url":"https://arxiv.org/abs/2001.09694v4","paper_title":"Retrospective Reader for Machine Reading Comprehension","code":"https://github.com/cooelf/AwesomeMRC","n_code_links":2,"syntology":null},{"rank_in_archive_order":66,"model":"ALBERT (single model)","metrics":{"EM":"88.107","F1":"90.902"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":67,"model":"ALBERT (single model)","metrics":{"EM":"88.107","F1":"90.902"},"uses_additional_data":false,"paper_date":"2019-09-26","paper":"/paper/albert-a-lite-bert-for-self-supervised","paper_url":"https://arxiv.org/abs/1909.11942v6","paper_title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations","code":"https://github.com/huggingface/transformers","n_code_links":48,"syntology":{"n_ran":46,"n_unverified":80,"n_samples":126,"n_pointer_only_licence":22}},{"rank_in_archive_order":68,"model":"{MTL} (single model)","metrics":{"EM":"88.107","F1":"90.902"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":69,"model":"MTL (single model)","metrics":{"EM":"88.107","F1":"90.902"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":70,"model":"albert_with_tricks (single)","metrics":{"EM":"88.073","F1":"91.179"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":71,"model":"ALBert (single-model)","metrics":{"EM":"88.050","F1":"91.036"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":72,"model":"XLNet + SG-Net Verifier (ensemble)","metrics":{"EM":"88.050","F1":"90.645"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":73,"model":"DeBERTalarge","metrics":{"EM":"88.0","F1":"90.7"},"uses_additional_data":false,"paper_date":"2020-06-05","paper":"/paper/deberta-decoding-enhanced-bert-with","paper_url":"https://arxiv.org/abs/2006.03654v6","paper_title":"DeBERTa: Decoding-enhanced BERT with Disentangled Attention","code":"https://github.com/huggingface/transformers","n_code_links":14,"syntology":{"n_ran":4,"n_unverified":9,"n_samples":13,"n_pointer_only_licence":3}},{"rank_in_archive_order":74,"model":"ALBERT 1.1 Th (single model)","metrics":{"EM":"87.994","F1":"90.944"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":75,"model":"SkERT-Large (single model)","metrics":{"EM":"87.994","F1":"90.944"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":76,"model":"albert+KD+transfer+twopass (single)","metrics":{"EM":"87.949","F1":"90.818"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":77,"model":"XLNet (single model)","metrics":{"EM":"87.926","F1":"90.689"},"uses_additional_data":false,"paper_date":"2019-06-19","paper":"/paper/xlnet-generalized-autoregressive-pretraining","paper_url":"https://arxiv.org/abs/1906.08237v2","paper_title":"XLNet: Generalized Autoregressive Pretraining for Language Understanding","code":"https://github.com/huggingface/transformers","n_code_links":27,"syntology":{"n_ran":10,"n_unverified":14,"n_samples":24,"n_pointer_only_licence":3}},{"rank_in_archive_order":78,"model":"ALBERT+RL (single model)","metrics":{"EM":"87.870","F1":"90.823"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":79,"model":"ALBERT+Entailment DA Verifier (single model)","metrics":{"EM":"87.847","F1":"91.265"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":80,"model":"Tuned ALBERT (single model)","metrics":{"EM":"87.847","F1":"90.532"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":81,"model":"albert_xxlarge (single model)","metrics":{"EM":"87.802","F1":"90.872"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":82,"model":"ALBERT 1.1 (single model)","metrics":{"EM":"87.700","F1":"90.588"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":83,"model":"LUKE (single model)","metrics":{"EM":"87.429","F1":"90.163"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":84,"model":"LUKE (single model)","metrics":{"EM":"87.429","F1":"90.163"},"uses_additional_data":false,"paper_date":"2020-10-02","paper":"/paper/luke-deep-contextualized-entity","paper_url":"https://arxiv.org/abs/2010.01057v1","paper_title":"LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention","code":"https://github.com/huggingface/transformers","n_code_links":9,"syntology":{"n_ran":3,"n_unverified":7,"n_samples":10,"n_pointer_only_licence":0}},{"rank_in_archive_order":85,"model":"XLNet + SG-Net Verifier++ (single model)","metrics":{"EM":"87.238","F1":"90.071"},"uses_additional_data":false,"paper_date":"2019-08-14","paper":"/paper/sg-net-syntax-guided-machine-reading","paper_url":"https://arxiv.org/abs/1908.05147v3","paper_title":"SG-Net: Syntax-Guided Machine Reading Comprehension","code":"https://github.com/cooelf/SG-Net","n_code_links":1,"syntology":null},{"rank_in_archive_order":86,"model":"UPM (single model)","metrics":{"EM":"87.193","F1":"89.934"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":87,"model":"BERT + DAE + AoA (ensemble)","metrics":{"EM":"87.147","F1":"89.474"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":88,"model":"XLNet + SG-Net Verifier (single model)","metrics":{"EM":"87.046","F1":"89.899"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":89,"model":"RoBERTa+Verify (ensemble)","metrics":{"EM":"86.933","F1":"90.037"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":90,"model":"RoBERTa (single model)","metrics":{"EM":"86.820","F1":"89.795"},"uses_additional_data":false,"paper_date":"2019-07-26","paper":"/paper/roberta-a-robustly-optimized-bert-pretraining","paper_url":"https://arxiv.org/abs/1907.11692v1","paper_title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","code":"https://github.com/huggingface/transformers","n_code_links":67,"syntology":{"n_ran":22,"n_unverified":26,"n_samples":48,"n_pointer_only_licence":23}},{"rank_in_archive_order":91,"model":"RoBERTa (single model)","metrics":{"EM":"86.820","F1":"89.795"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":92,"model":"BERT + ConvLSTM + MTL + Verifier (ensemble)","metrics":{"EM":"86.730","F1":"89.286"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":93,"model":"BERT + N-Gram Masking + Synthetic Self-Training (ensemble)","metrics":{"EM":"86.673","F1":"89.147"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":94,"model":"RoBERTa+Span (ensemble)","metrics":{"EM":"86.651","F1":"89.595"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":95,"model":"Xlnet+Verifier","metrics":{"EM":"86.594","F1":"89.082"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":96,"model":"Xlnet+Verifier(single model)","metrics":{"EM":"86.572","F1":"89.063"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":97,"model":"Xlnet+Verifier (single model)","metrics":{"EM":"86.572","F1":"89.063"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":98,"model":"RoBERTa+Verify (single model)","metrics":{"EM":"86.448","F1":"89.586"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":99,"model":"XLNET-123 (single model)","metrics":{"EM":"86.436","F1":"89.086"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":100,"model":"XLNET-V2-123+ (single model)","metrics":{"EM":"86.403","F1":"89.148"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":101,"model":"XLNet (single model)","metrics":{"EM":"86.346","F1":"89.133"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":102,"model":"SG-Net (ensemble)","metrics":{"EM":"86.211","F1":"88.848"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":103,"model":"SG-Net (ensemble)","metrics":{"EM":"86.211","F1":"88.848"},"uses_additional_data":false,"paper_date":"2019-08-14","paper":"/paper/sg-net-syntax-guided-machine-reading","paper_url":"https://arxiv.org/abs/1908.05147v3","paper_title":"SG-Net: Syntax-Guided Machine Reading Comprehension","code":"https://github.com/cooelf/SG-Net","n_code_links":1,"syntology":null},{"rank_in_archive_order":104,"model":"BERT++(ensemble)","metrics":{"EM":"86.166","F1":"88.886"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":105,"model":"SemBERT(ensemble)","metrics":{"EM":"86.166","F1":"88.886"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":106,"model":"SemBERT(ensemble)","metrics":{"EM":"86.166","F1":"88.886"},"uses_additional_data":false,"paper_date":"2019-09-05","paper":"/paper/semantics-aware-bert-for-language","paper_url":"https://arxiv.org/abs/1909.02209v3","paper_title":"Semantics-aware BERT for Language Understanding","code":"https://github.com/cooelf/SemBERT","n_code_links":1,"syntology":{"n_ran":4,"n_unverified":8,"n_samples":12,"n_pointer_only_licence":0}},{"rank_in_archive_order":107,"model":"SemBERT (ensemble)","metrics":{"EM":"86.166","F1":"88.886"},"uses_additional_data":false,"paper_date":"2019-09-05","paper":"/paper/semantics-aware-bert-for-language","paper_url":"https://arxiv.org/abs/1909.02209v3","paper_title":"Semantics-aware BERT for Language Understanding","code":"https://github.com/cooelf/SemBERT","n_code_links":1,"syntology":{"n_ran":4,"n_unverified":8,"n_samples":12,"n_pointer_only_licence":0}},{"rank_in_archive_order":108,"model":"Enhanced Albert+Verifier (ensemble)","metrics":{"EM":"86.098","F1":"89.634"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":109,"model":"BERT + DAE + AoA (single model)","metrics":{"EM":"85.884","F1":"88.621"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":110,"model":"RoBERTa+Span (single model)","metrics":{"EM":"85.872","F1":"88.989"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":111,"model":"RoBERTa-Large (ensemble model)","metrics":{"EM":"85.872","F1":"88.793"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":112,"model":"BNDVnet (ensemble model)","metrics":{"EM":"85.850","F1":"88.449"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":113,"model":"Unnamed submission by guo","metrics":{"EM":"85.838","F1":"88.921"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":114,"model":"BERTSP(single model)","metrics":{"EM":"85.838","F1":"88.921"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":115,"model":"BERTSP (single model)","metrics":{"EM":"85.838","F1":"88.921"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":116,"model":"Enhanced Albert+Verifier3 (ensemble)","metrics":{"EM":"85.827","F1":"89.778"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":117,"model":"SENSEFORTH + XLNet (single model)","metrics":{"EM":"85.827","F1":"88.699"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":118,"model":"SpanBERT (single model)","metrics":{"EM":"85.748","F1":"88.709"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":119,"model":"SemBERT (ensemble model)","metrics":{"EM":"85.703","F1":"88.400"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":120,"model":"SpanBERT","metrics":{"EM":"85.7","F1":"88.7"},"uses_additional_data":false,"paper_date":"2019-07-24","paper":"/paper/spanbert-improving-pre-training-by","paper_url":"https://arxiv.org/abs/1907.10529v3","paper_title":"SpanBERT: Improving Pre-training by Representing and Predicting Spans","code":"https://github.com/facebookresearch/SpanBERT","n_code_links":6,"syntology":{"n_ran":3,"n_unverified":12,"n_samples":15,"n_pointer_only_licence":4}},{"rank_in_archive_order":121,"model":"BertCNN (single model)","metrics":{"EM":"85.240","F1":"87.901"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":122,"model":"SG-Net (single model)","metrics":{"EM":"85.229","F1":"87.926"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":123,"model":"SG-Net (single model)","metrics":{"EM":"85.229","F1":"87.926"},"uses_additional_data":false,"paper_date":"2019-08-14","paper":"/paper/sg-net-syntax-guided-machine-reading","paper_url":"https://arxiv.org/abs/1908.05147v3","paper_title":"SG-Net: Syntax-Guided Machine Reading Comprehension","code":"https://github.com/cooelf/SG-Net","n_code_links":1,"syntology":null},{"rank_in_archive_order":124,"model":"RoBERTa-Large (single model)","metrics":{"EM":"85.173","F1":"88.425"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":125,"model":"BERT + N-Gram Masking + Synthetic Self-Training (single model)","metrics":{"EM":"85.150","F1":"87.715"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":126,"model":"BERT + MMFT + ADA (ensemble)","metrics":{"EM":"85.082","F1":"87.615"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":127,"model":"BNDVnet (single model)","metrics":{"EM":"85.003","F1":"87.833"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":128,"model":"BERT + ConvLSTM + MTL + Verifier (single model)","metrics":{"EM":"84.924","F1":"88.204"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":129,"model":"Insight-baseline-BERT (single model)","metrics":{"EM":"84.834","F1":"87.644"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":130,"model":"BertSpan (ensemble)","metrics":{"EM":"84.823","F1":"87.489"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":131,"model":"SemBERT (single model)","metrics":{"EM":"84.800","F1":"87.864"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":132,"model":"SemBERT (single model)","metrics":{"EM":"84.800","F1":"87.864"},"uses_additional_data":false,"paper_date":"2019-09-05","paper":"/paper/semantics-aware-bert-for-language","paper_url":"https://arxiv.org/abs/1909.02209v3","paper_title":"Semantics-aware BERT for Language Understanding","code":"https://github.com/cooelf/SemBERT","n_code_links":1,"syntology":{"n_ran":4,"n_unverified":8,"n_samples":12,"n_pointer_only_licence":0}},{"rank_in_archive_order":133,"model":"Hanvon_model(single model)","metrics":{"EM":"84.721","F1":"87.117"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":134,"model":"Hanvon_model (single model)","metrics":{"EM":"84.721","F1":"87.117"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":135,"model":"xlnet","metrics":{"EM":"84.642","F1":"88.000"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":136,"model":"xlnet(single model)","metrics":{"EM":"84.642","F1":"88.000"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":137,"model":"xlnet (single model)","metrics":{"EM":"84.642","F1":"88.000"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":138,"model":"BERT++(single model)","metrics":{"EM":"84.620","F1":"87.625"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":139,"model":"BERT + Synthetic Self-Training (ensemble)","metrics":{"EM":"84.292","F1":"86.967"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":140,"model":"BERT + Multiple-CNN (ensemble)(Kyonggi University","metrics":{"EM":"84.202","F1":"86.767"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":141,"model":"BERT + Multiple-CNN (ensemble)","metrics":{"EM":"84.202","F1":"86.767"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":142,"model":"RoberTa Parallel Adapters Singl","metrics":{"EM":"84.123","F1":"87.013"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":143,"model":"RoberTa Parallel Adapters Single","metrics":{"EM":"84.123","F1":"87.013"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":144,"model":"RoberTa+Parallel+Adapters (single model)","metrics":{"EM":"84.123","F1":"87.013"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":145,"model":"SemNet (single model)","metrics":{"EM":"83.819","F1":"86.669"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":146,"model":"Tuned BERT-1seq Large Cased (single model)","metrics":{"EM":"83.751","F1":"86.594"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":147,"model":"BERT finetune baseline (ensemble)","metrics":{"EM":"83.536","F1":"86.096"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":148,"model":"SynNet (single model)","metrics":{"EM":"83.525","F1":"86.222"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":149,"model":"Lunet + Verifier + BERT (ensemble)","metrics":{"EM":"83.469","F1":"86.043"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":150,"model":"PAML+BERT (ensemble model)","metrics":{"EM":"83.457","F1":"86.122"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":151,"model":"Unnamed submission by Senseforth_AI","metrics":{"EM":"83.142","F1":"85.873"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":152,"model":"SENSEFORTH + BERT","metrics":{"EM":"83.142","F1":"85.873"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":153,"model":"Bert-raw (ensemble)","metrics":{"EM":"83.119","F1":"85.510"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":154,"model":"BERT with Something (ensemble)","metrics":{"EM":"83.051","F1":"85.737"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":155,"model":"BERT + MMFT + ADA (single model)","metrics":{"EM":"83.040","F1":"85.892"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":156,"model":"Lunet + Verifier + BERT (single model)","metrics":{"EM":"82.995","F1":"86.035"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":157,"model":"BERT + Synthetic Self-Training (single model)","metrics":{"EM":"82.972","F1":"85.810"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":158,"model":"mgrc_l (single model)","metrics":{"EM":"82.961","F1":"86.075"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":159,"model":"ATB (single model)","metrics":{"EM":"82.882","F1":"86.002"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":160,"model":"Tuned BERT Large Cased (single model)","metrics":{"EM":"82.803","F1":"85.863"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":161,"model":"BERT-Base + QA Pre-training (single model)","metrics":{"EM":"82.724","F1":"85.491"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":162,"model":"BERT + NeurQuRI (ensemble)","metrics":{"EM":"82.713","F1":"85.584"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":163,"model":"PAML+BERT (single model)","metrics":{"EM":"82.577","F1":"85.603"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":164,"model":"Unnamed submission by cooelf","metrics":{"EM":"82.431","F1":"85.178"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":165,"model":"AoA + DA + BERT (ensemble)","metrics":{"EM":"82.374","F1":"85.310"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":166,"model":"BART + Adapters + Lohfink-Rossi-Leaveout (single-model)","metrics":{"EM":"82.306","F1":"85.670"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":167,"model":"BERT finetune baseline (single model)","metrics":{"EM":"82.126","F1":"84.820"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":168,"model":"Candi-Net+BERT (ensemble)","metrics":{"EM":"82.126","F1":"84.624"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":169,"model":"BERT-Base PMI-Masking Additional Data (single model)","metrics":{"EM":"82.024","F1":"84.854"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":170,"model":"BERT_s (single model)","metrics":{"EM":"81.979","F1":"84.846"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":171,"model":"Unnamed submission by zw10","metrics":{"EM":"81.731","F1":"84.862"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":172,"model":"BERT-large+UBFT (single model)","metrics":{"EM":"81.573","F1":"84.535"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":173,"model":"AoA + DA + BERT (single model)","metrics":{"EM":"81.178","F1":"84.251"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":174,"model":"BERT with Something (single model)","metrics":{"EM":"81.110","F1":"84.386"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":175,"model":"BERT-Base PMI-Masking (single model)","metrics":{"EM":"80.896","F1":"83.604"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":176,"model":"BERT + UnAnsQ (single model)","metrics":{"EM":"80.749","F1":"83.851"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":177,"model":"BERT + AL (single model)","metrics":{"EM":"80.715","F1":"83.827"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":178,"model":"BERT + NeurQuRI (single model)","metrics":{"EM":"80.591","F1":"83.391"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":179,"model":"BERTlarge (ensemble)","metrics":{"EM":"80.456","F1":"83.509"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":180,"model":"Bert","metrics":{"EM":"80.422","F1":"83.118"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":181,"model":"Bert-raw (single)","metrics":{"EM":"80.411","F1":"83.457"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":182,"model":"Candi-Net+BERT (single model)","metrics":{"EM":"80.388","F1":"82.908"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":183,"model":"PMI-Masking Additional Data Random Baseline (single model)","metrics":{"EM":"80.377","F1":"83.262"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":184,"model":"Unnamed submission by zw4","metrics":{"EM":"80.354","F1":"83.329"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":185,"model":"{Bert-Span} (single model)","metrics":{"EM":"80.354","F1":"83.329"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":186,"model":"Bert-raw (single model)","metrics":{"EM":"80.343","F1":"83.243"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":187,"model":"PMI-Masking Pure-PMI (single model)","metrics":{"EM":"80.241","F1":"83.175"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":188,"model":"BISAN-CC (single model)","metrics":{"EM":"80.208","F1":"83.149"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":189,"model":"ST_bl","metrics":{"EM":"80.140","F1":"82.962"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":190,"model":"PwP+BERT (single model)","metrics":{"EM":"80.117","F1":"83.189"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":191,"model":"PMI-Masking Random Baseline (single model)","metrics":{"EM":"80.038","F1":"82.796"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":192,"model":"BERT + UDA (single model)","metrics":{"EM":"80.005","F1":"83.208"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":193,"model":"PMI-Masking Additional Data Pure-PMI (single model)","metrics":{"EM":"79.993","F1":"83.039"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":194,"model":"Original BERT Large Cased (single model)","metrics":{"EM":"79.971","F1":"83.266"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":195,"model":"bert (single model)","metrics":{"EM":"79.971","F1":"83.184"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":196,"model":"BERT + Sparse-Transformer","metrics":{"EM":"79.948","F1":"83.023"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":197,"model":"Insight-baseline (single model)","metrics":{"EM":"79.779","F1":"83.099"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":198,"model":"NEXYS_BASE (single model)","metrics":{"EM":"79.779","F1":"82.912"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":199,"model":"BERT uncased (single model)","metrics":{"EM":"79.745","F1":"83.020"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":200,"model":"{bert-finetuning} (single model)","metrics":{"EM":"79.632","F1":"82.852"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":201,"model":"L6Net + BERT (single model)","metrics":{"EM":"79.181","F1":"82.259"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":202,"model":"Fusion Adapters TriviaQA NQ Singl","metrics":{"EM":"78.933","F1":"81.863"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":203,"model":"RoberTa Fusion Adapters Single","metrics":{"EM":"78.933","F1":"81.863"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":204,"model":"RoberTa+Fusion+Adapters (single model)","metrics":{"EM":"78.933","F1":"81.863"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":205,"model":"{Anonymous} (single model)","metrics":{"EM":"78.876","F1":"82.524"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":206,"model":"BERT + WIAN (ensemble)","metrics":{"EM":"78.650","F1":"81.497"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":207,"model":"BERTlarge (single model)","metrics":{"EM":"78.650","F1":"81.474"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":208,"model":"AMBERT (single model)","metrics":{"EM":"78.594","F1":"81.445"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":209,"model":"BISAN (single model)","metrics":{"EM":"78.481","F1":"81.531"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":210,"model":"BERT-Large-Cased (single model)","metrics":{"EM":"78.357","F1":"81.500"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":211,"model":"BERT-Large-Cased","metrics":{"EM":"78.357","F1":"81.500"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":212,"model":"BERT+AC(single model)","metrics":{"EM":"78.052","F1":"81.174"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":213,"model":"BERT+AC (single model)","metrics":{"EM":"78.052","F1":"81.174"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":214,"model":"BERT (single model)","metrics":{"EM":"77.319","F1":"80.310"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":215,"model":"TriviaQA Adapter Single Tune","metrics":{"EM":"77.262","F1":"80.258"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":216,"model":"RoberTa Adapter Single","metrics":{"EM":"77.262","F1":"80.258"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":217,"model":"RoberTa+Adapter (single model)","metrics":{"EM":"77.262","F1":"80.258"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":218,"model":"SLQA+BERT (single model)","metrics":{"EM":"77.003","F1":"80.209"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":219,"model":"AMBERT-H (single model)","metrics":{"EM":"76.710","F1":"79.659"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":220,"model":"AMBERT-S (single model)","metrics":{"EM":"76.563","F1":"79.776"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":221,"model":"synss (single model )","metrics":{"EM":"76.055","F1":"79.329"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":222,"model":"synss (single model)","metrics":{"EM":"76.055","F1":"79.329"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":223,"model":"BERT-Base-L (single model)","metrics":{"EM":"75.457","F1":"78.232"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":224,"model":"mgrc","metrics":{"EM":"75.344","F1":"78.381"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":225,"model":"BERT-Base-V (single model)","metrics":{"EM":"75.073","F1":"77.805"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":226,"model":"MIR-MRC(F-Net) (single model)","metrics":{"EM":"74.791","F1":"77.988"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":227,"model":"MIR-MRC (F-Net)","metrics":{"EM":"74.791","F1":"77.988"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":228,"model":"BERT-Base-DT (single model)","metrics":{"EM":"74.769","F1":"77.706"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":229,"model":"ARSG-BERT (single model)","metrics":{"EM":"74.746","F1":"78.227"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":230,"model":"BERT-Base-V2","metrics":{"EM":"74.656","F1":"77.404"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":231,"model":"BERT-Base-DP (single model)","metrics":{"EM":"74.577","F1":"77.464"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":232,"model":"{BERTcw} (single model)","metrics":{"EM":"74.385","F1":"77.308"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":233,"model":"BERT-Base-Add (single model)","metrics":{"EM":"74.329","F1":"77.396"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":234,"model":"nlnet (single model)","metrics":{"EM":"74.272","F1":"77.052"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":235,"model":"batch2 (single model)","metrics":{"EM":"73.742","F1":"76.858"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":236,"model":"MMIPN","metrics":{"EM":"73.505","F1":"76.424"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":237,"model":"BERT-Base-Baseline (single model)","metrics":{"EM":"73.302","F1":"76.284"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":238,"model":"BERT-Base (single model)","metrics":{"EM":"73.099","F1":"76.236"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":239,"model":"ICL_MODEL(ensemble)","metrics":{"EM":"72.884","F1":"76.217"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":240,"model":"ICL_MODEL (ensemble)","metrics":{"EM":"72.884","F1":"76.217"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":241,"model":"YARCS (ensemble)","metrics":{"EM":"72.670","F1":"75.507"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":242,"model":"BERT-base","metrics":{"EM":"72.072","F1":"75.513"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":243,"model":"BERTBase (single model)","metrics":{"EM":"72.072","F1":"75.513"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":244,"model":"Reinforced Mnemonic Reader + Answer Verifier (single model)","metrics":{"EM":"71.767","F1":"74.295"},"uses_additional_data":false,"paper_date":"2018-08-17","paper":"/paper/read-verify-machine-reading-comprehension","paper_url":"http://arxiv.org/abs/1808.05759v5","paper_title":"Read + Verify: Machine Reading Comprehension with Unanswerable Questions","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":245,"model":"BERT-Base (single)","metrics":{"EM":"71.699","F1":"74.430"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":246,"model":"BERT+Answer Verifier (single model)","metrics":{"EM":"71.666","F1":"75.457"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":247,"model":"SLQA+ (single model)","metrics":{"EM":"71.462","F1":"74.434"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":248,"model":"Unet (ensemble)","metrics":{"EM":"71.417","F1":"74.869"},"uses_additional_data":false,"paper_date":"2018-10-12","paper":"/paper/u-net-machine-reading-comprehension-with","paper_url":"http://arxiv.org/abs/1810.06638v1","paper_title":"U-Net: Machine Reading Comprehension with Unanswerable Questions","code":"https://github.com/FudanNLP/UNet","n_code_links":1,"syntology":null},{"rank_in_archive_order":249,"model":"SAN (ensemble model)","metrics":{"EM":"71.316","F1":"73.704"},"uses_additional_data":false,"paper_date":"2017-12-10","paper":"/paper/stochastic-answer-networks-for-machine","paper_url":"http://arxiv.org/abs/1712.03556v2","paper_title":"Stochastic Answer Networks for Machine Reading Comprehension","code":"https://github.com/kevinduh/san_mrc","n_code_links":6,"syntology":null},{"rank_in_archive_order":250,"model":"HYDRA_BERT (single model)","metrics":{"EM":"71.293","F1":"74.578"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":251,"model":"{BERT-base} (single-model)","metrics":{"EM":"70.763","F1":"74.449"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":252,"model":"FusionNet++ (ensemble)","metrics":{"EM":"70.300","F1":"72.484"},"uses_additional_data":false,"paper_date":"2017-11-16","paper":"/paper/fusionnet-fusing-via-fully-aware-attention","paper_url":"http://arxiv.org/abs/1711.07341v2","paper_title":"FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension","code":"https://github.com/momohuang/FusionNet-NLI","n_code_links":3,"syntology":null},{"rank_in_archive_order":253,"model":"Multi-Level Attention Fusion(MLAF) (single model)","metrics":{"EM":"69.476","F1":"72.857"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":254,"model":"Multi-Level Attention Fusion (MLAF)","metrics":{"EM":"69.476","F1":"72.857"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":255,"model":"Unet (single model)","metrics":{"EM":"69.262","F1":"72.642"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":256,"model":"DocQA + NeurQuRI (single model)","metrics":{"EM":"68.766","F1":"71.662"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":257,"model":"SAN (single model)","metrics":{"EM":"68.653","F1":"71.439"},"uses_additional_data":false,"paper_date":"2017-12-10","paper":"/paper/stochastic-answer-networks-for-machine","paper_url":"http://arxiv.org/abs/1712.03556v2","paper_title":"Stochastic Answer Networks for Machine Reading Comprehension","code":"https://github.com/kevinduh/san_mrc","n_code_links":6,"syntology":null},{"rank_in_archive_order":258,"model":"KACTEIL-MRC(GFN-Net) (single model)","metrics":{"EM":"68.213","F1":"70.878"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":259,"model":"KACTEIL-MRC (GFN-Net)","metrics":{"EM":"68.213","F1":"70.878"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":260,"model":"BiDAF++ with pair2vec (single model)","metrics":{"EM":"68.021","F1":"71.583"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":261,"model":"VS^3-NET (single model)","metrics":{"EM":"67.897","F1":"70.884"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":262,"model":"EBB-Net (single model)","metrics":{"EM":"66.610","F1":"70.303"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":263,"model":"KakaoNet2 (single model)","metrics":{"EM":"65.719","F1":"69.381"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":264,"model":"BiDAF++ (single model)","metrics":{"EM":"65.651","F1":"68.866"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":265,"model":"abcNet (single model)","metrics":{"EM":"65.256","F1":"69.206"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":266,"model":"BiDAF + Self Attention + ELMo (single model)","metrics":{"EM":"63.372","F1":"66.251"},"uses_additional_data":false,"paper_date":"2018-02-15","paper":"/paper/deep-contextualized-word-representations","paper_url":"http://arxiv.org/abs/1802.05365v2","paper_title":"Deep contextualized word representations","code":"https://github.com/flairNLP/flair","n_code_links":46,"syntology":{"n_ran":23,"n_unverified":35,"n_samples":58,"n_pointer_only_licence":25}},{"rank_in_archive_order":267,"model":"BiDAF + Self Attention + ELMo (single model)","metrics":{"EM":"63.372","F1":"66.251"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":268,"model":"BSAE AddText (single model)","metrics":{"EM":"63.338","F1":"67.422"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":269,"model":"eeAttNet (single model)","metrics":{"EM":"63.327","F1":"66.633"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":270,"model":"BiDAF + Self Attention (single model)","metrics":{"EM":"59.332","F1":"62.305"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":271,"model":"BiDAF-No-Answer (single model)","metrics":{"EM":"59.174","F1":"62.093"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":272,"model":"BNA + SoftDrop (single model)","metrics":{"EM":"58.508","F1":"62.045"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":273,"model":"Tree-LSTM + BiDAF + ELMo (single model)","metrics":{"EM":"57.707","F1":"62.341"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":274,"model":"BNA + HardDrop (single model)","metrics":{"EM":"56.545","F1":"59.546"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":275,"model":"Unnamed submission by Simon","metrics":{"EM":"56.545","F1":"59.546"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":276,"model":"Anonymous (single model)","metrics":{"EM":"49.695","F1":"49.701"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":277,"model":"{FOO} (single model)","metrics":{"EM":"48.883","F1":"48.883"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":278,"model":"Bert Large Sentence (Single Model)","metrics":{"EM":"48.804","F1":"48.815"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":279,"model":"XLNet + DAAF + BERTverifier (ensemble)","metrics":{"EM":"44.945","F1":"47.994"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":280,"model":"TSAN","metrics":{"EM":"40.397","F1":"43.213"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":281,"model":"","metrics":{"EM":"27.217","F1":"29.597"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":282,"model":"CHECK SYSTEM (single model)","metrics":{"EM":"4.830","F1":"5.920"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":283,"model":"5cls_squad1_fai","metrics":{"EM":"0.068","F1":"3.971"},"uses_additional_data":false,"paper_date":null,"paper":null,"paper_url":null,"paper_title":"","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":284,"model":"LUKE 483M","metrics":{"F1":"90.2"},"uses_additional_data":false,"paper_date":"2020-10-02","paper":"/paper/luke-deep-contextualized-entity","paper_url":"https://arxiv.org/abs/2010.01057v1","paper_title":"LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention","code":"https://github.com/huggingface/transformers","n_code_links":9,"syntology":{"n_ran":3,"n_unverified":7,"n_samples":10,"n_pointer_only_licence":0}},{"rank_in_archive_order":285,"model":"Ensemble ALBERT","metrics":{"F1":"90.123"},"uses_additional_data":false,"paper_date":"2021-10-19","paper":"/paper/ensemble-albert-on-squad-2-0","paper_url":"https://arxiv.org/abs/2110.09665v1","paper_title":"Ensemble ALBERT on SQuAD 2.0","code":"https://github.com/Shilun-Allan-Li/Ensemble-ALBERT-on-SQuAD-2.0","n_code_links":1,"syntology":null},{"rank_in_archive_order":286,"model":"gMLP-large","metrics":{"F1":"78.3"},"uses_additional_data":false,"paper_date":"2021-05-17","paper":"/paper/pay-attention-to-mlps","paper_url":"https://arxiv.org/abs/2105.08050v2","paper_title":"Pay Attention to MLPs","code":"https://github.com/labmlai/annotated_deep_learning_paper_implementations","n_code_links":20,"syntology":{"n_ran":34,"n_unverified":10,"n_samples":44,"n_pointer_only_licence":11}}],"since_archive":{"present":false,"note":"No Syntology-extracted rows are published in this build."},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per row: N of M harvested code samples from that row's paper executed on a synthesized fixture; the other M-N are unverified. Not a reproduction of the row's number; not a correctness claim. n_pointer_only_licence counts samples the site points at rather than redistributes (a licence axis, independent of ran/unverified).","rows_with_graph_line":13,"rows_with_any_sample_ran":13,"distinct_papers_with_graph_line":9,"distinct_papers_with_any_sample_ran":9,"samples_over_distinct_papers":{"n_ran":149,"n_unverified":201,"n_samples":350,"n_pointer_only_licence":91,"note":"each paper (arXiv id) counted once, however many rows it is behind; this is the page-level figure"},"samples_row_weighted":{"n_ran":206,"n_unverified":304,"n_samples":510,"n_pointer_only_licence":113,"note":"row-weighted: a paper behind several rows is counted once per row; inflated relative to samples_over_distinct_papers by design, kept for readers summing the per-row syntology blocks"}}}