{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/xlnet-generalized-autoregressive-pretraining","title":"XLNet: Generalized Autoregressive Pretraining for Language Understanding","arxiv_id":"1906.08237","date":"2019-06-19","proceeding":"NeurIPS 2019 12","authors":["Zhilin Yang","Zihang Dai","Yiming Yang","Jaime Carbonell","Ruslan Salakhutdinov","Quoc V. Le"],"abstract":"With the capability of modeling bidirectional contexts, denoising autoencoding based pretraining like BERT achieves better performance than pretraining approaches based on autoregressive language modeling. However, relying on corrupting the input with masks, BERT neglects dependency between the masked positions and suffers from a pretrain-finetune discrepancy. In light of these pros and cons, we propose XLNet, a generalized autoregressive pretraining method that (1) enables learning bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order and (2) overcomes the limitations of BERT thanks to its autoregressive formulation. Furthermore, XLNet integrates ideas from Transformer-XL, the state-of-the-art autoregressive model, into pretraining. Empirically, under comparable experiment settings, XLNet outperforms BERT on 20 tasks, often by a large margin, including question answering, natural language inference, sentiment analysis, and document ranking.","url_abs":"https://arxiv.org/abs/1906.08237v2","url_pdf":"https://arxiv.org/pdf/1906.08237v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/zihangdai/xlnet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/2miatran/Natural-Language-Processing","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/NathanDuran/Sentence-Encoding-for-DA-Classification","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/SambhawDrag/XLNet.jl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/chesterdu/contrastive_summary","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/cuhksz-nlp/SAPar","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/facebookresearch/anli","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/fanchenyou/transformer-study","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/graykode/xlnet-Pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/https-seyhan/BugAI","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/huggingface/transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/huggingface/xlnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/jonahwinninghoff/Text-Summarization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/joshuaWang-bit/Textclassification-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/kaushaltrivedi/fast-bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/listenviolet/XLNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/pauldevos/python-notes","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/samwisegamjeee/pytorch-transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/tomgoter/nlp_finalproject","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/utterworks/fast-bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/zaradana/Fast_BERT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/MS-P3/code7/tree/main/xlnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/MindCode-4/code-5/tree/main/xlnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/language_model/xlnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"paddle","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/lvyufeng/bert4ms/blob/master/bert4ms/models/xlnet.py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/pwc-1/Paper-9/tree/main/1/xlnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"xlnet-generalized-autoregressive-pretraining","repo_url":"https://github.com/pwc-1/Paper-9/tree/main/5/xlnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"audio-question-answering","task_name":"Audio Question Answering"},{"task_slug":"chinese-reading-comprehension","task_name":"Chinese Reading Comprehension"},{"task_slug":"document-ranking","task_name":"Document Ranking"},{"task_slug":"humor-detection","task_name":"Humor Detection"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"paraphrase-identification","task_name":"Paraphrase Identification"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"reading-comprehension","task_name":"Reading Comprehension"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"text-classification","task_name":"Text Classification"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"adaptive-input-representations","method_name":"Adaptive Input Representations"},{"method_slug":"adaptive-softmax","method_name":"Adaptive Softmax"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sentencepiece","method_name":"SentencePiece"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer-xl","method_name":"Transformer-XL"},{"method_slug":"variational-dropout","method_name":"Variational Dropout"},{"method_slug":"xlnet","method_name":"XLNet"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/document-ranking-on-clueweb09-b","task":"Document Ranking","dataset":"ClueWeb09-B","model":"XLNet","rank_in_archive_order":1,"of":1,"metrics":{"ERR@20":"20.28","nDCG@20":"31.10"},"uses_additional_data":false},{"leaderboard":"/sota/humor-detection-on-200k-short-texts-for-humor-1","task":"Humor Detection","dataset":"200k Short Texts for Humor Detection","model":"XLNet Large Cased","rank_in_archive_order":2,"of":6,"metrics":{"F1-score":"0.920"},"uses_additional_data":false},{"leaderboard":"/sota/linguistic-acceptability-on-cola","task":"Linguistic Acceptability","dataset":"CoLA","model":"XLNet (single model)","rank_in_archive_order":15,"of":43,"metrics":{"Accuracy":"69%"},"uses_additional_data":false},{"leaderboard":"/sota/natural-language-inference-on-anli-test","task":"Natural Language Inference","dataset":"ANLI test","model":"XLNet (Large)","rank_in_archive_order":7,"of":25,"metrics":{"A1":"70.3","A2":"50.9","A3":"49.4"},"uses_additional_data":true},{"leaderboard":"/sota/natural-language-inference-on-multinli","task":"Natural Language Inference","dataset":"MultiNLI","model":"XLNet (single model)","rank_in_archive_order":10,"of":67,"metrics":{"Matched":"90.8"},"uses_additional_data":false},{"leaderboard":"/sota/natural-language-inference-on-qnli","task":"Natural Language Inference","dataset":"QNLI","model":"XLNet (single model)","rank_in_archive_order":11,"of":43,"metrics":{"Accuracy":"94.9%"},"uses_additional_data":false},{"leaderboard":"/sota/natural-language-inference-on-rte","task":"Natural Language Inference","dataset":"RTE","model":"XLNet (single model)","rank_in_archive_order":24,"of":90,"metrics":{"Accuracy":"85.9%"},"uses_additional_data":false},{"leaderboard":"/sota/natural-language-inference-on-wnli","task":"Natural Language Inference","dataset":"WNLI","model":"XLNet","rank_in_archive_order":4,"of":23,"metrics":{"Accuracy":"92.5"},"uses_additional_data":false},{"leaderboard":"/sota/paraphrase-identification-on-quora-question","task":"Paraphrase Identification","dataset":"Quora Question Pairs","model":"XLNet-Large (ensemble)","rank_in_archive_order":7,"of":31,"metrics":{"Accuracy":"90.3","F1":"74.2"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-quora-question-pairs","task":"Question Answering","dataset":"Quora Question Pairs","model":"XLNet (single model)","rank_in_archive_order":1,"of":19,"metrics":{"Accuracy":"92.3%"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-race","task":"Question Answering","dataset":"RACE","model":"XLNet","rank_in_archive_order":1,"of":7,"metrics":{"RACE":"81.75","RACE-m":"85.45"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-squad11","task":"Question Answering","dataset":"SQuAD1.1","model":"XLNet (single model)","rank_in_archive_order":6,"of":213,"metrics":{"EM":"89.898","F1":"95.080","Hardware Burden":"46449G"},"uses_additional_data":true},{"leaderboard":"/sota/question-answering-on-squad11-dev","task":"Question Answering","dataset":"SQuAD1.1 dev","model":"XLNet (single model)","rank_in_archive_order":4,"of":55,"metrics":{"EM":"89.7","F1":"95.1"},"uses_additional_data":true},{"leaderboard":"/sota/question-answering-on-squad20","task":"Question Answering","dataset":"SQuAD2.0","model":"XLNet (single model)","rank_in_archive_order":77,"of":286,"metrics":{"EM":"87.926","F1":"90.689"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-squad20-dev","task":"Question Answering","dataset":"SQuAD2.0 dev","model":"XLNet (single model)","rank_in_archive_order":1,"of":13,"metrics":{"EM":"87.9","F1":"90.6"},"uses_additional_data":false},{"leaderboard":"/sota/reading-comprehension-on-race","task":"Reading Comprehension","dataset":"RACE","model":"XLNet","rank_in_archive_order":11,"of":24,"metrics":{"Accuracy (High)":"84.0","Accuracy (Middle)":"88.6"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-textual-similarity-on-mrpc","task":"Semantic Textual Similarity","dataset":"MRPC","model":"XLNet (single model)","rank_in_archive_order":9,"of":45,"metrics":{"Accuracy":"90.8%"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-textual-similarity-on-sts-benchmark","task":"Semantic Textual Similarity","dataset":"STS Benchmark","model":"XLNet (single model)","rank_in_archive_order":6,"of":66,"metrics":{"Pearson Correlation":"0.925"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-textual-similarity-on-senteval","task":"Semantic Textual Similarity","dataset":"SentEval","model":"XLNet-Large","rank_in_archive_order":6,"of":6,"metrics":{"MRPC":"93.0/90.7","SICK-E":"-","SICK-R":"-","STS":"91.6/91.1*"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-imdb","task":"Sentiment Analysis","dataset":"IMDb","model":"XLNet","rank_in_archive_order":3,"of":49,"metrics":{"Accuracy":"96.21"},"uses_additional_data":true},{"leaderboard":"/sota/sentiment-analysis-on-sst-2-binary","task":"Sentiment Analysis","dataset":"SST-2 Binary classification","model":"XLNet (single model)","rank_in_archive_order":7,"of":87,"metrics":{"Accuracy":"97"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-sst-2-binary","task":"Sentiment Analysis","dataset":"SST-2 Binary classification","model":"XLNet-Large (ensemble)","rank_in_archive_order":10,"of":87,"metrics":{"Accuracy":"96.8"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-yelp-binary","task":"Sentiment Analysis","dataset":"Yelp Binary classification","model":"XLNet","rank_in_archive_order":1,"of":20,"metrics":{"Error":"1.37"},"uses_additional_data":false},{"leaderboard":"/sota/sentiment-analysis-on-yelp-fine-grained","task":"Sentiment Analysis","dataset":"Yelp Fine-grained classification","model":"XLNet","rank_in_archive_order":1,"of":17,"metrics":{"Error":"27.05"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-ag-news","task":"Text Classification","dataset":"AG News","model":"XLNet","rank_in_archive_order":1,"of":24,"metrics":{"Error":"4.45"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-amazon-2","task":"Text Classification","dataset":"Amazon-2","model":"XLNet","rank_in_archive_order":1,"of":4,"metrics":{"Error":"2.11"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-amazon-5","task":"Text Classification","dataset":"Amazon-5","model":"XLNet","rank_in_archive_order":1,"of":3,"metrics":{"Error":"31.67"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-dbpedia","task":"Text Classification","dataset":"DBpedia","model":"XLNet","rank_in_archive_order":1,"of":21,"metrics":{"Error":"0.62"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-yelp-2","task":"Text Classification","dataset":"Yelp-2","model":"XLNet","rank_in_archive_order":1,"of":5,"metrics":{"Accuracy":"98.63%"},"uses_additional_data":false},{"leaderboard":"/sota/text-classification-on-yelp-5","task":"Text Classification","dataset":"Yelp-5","model":"XLNet","rank_in_archive_order":2,"of":7,"metrics":{"Accuracy":"72.95%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1906.08237","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1906.08237"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/transformers","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tomgoter/nlp_finalproject","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fanchenyou/transformer-study","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SambhawDrag/XLNet.jl","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2miatran/Natural-Language-Processing","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NathanDuran/Sentence-Encoding-for-DA-Classification","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/graykode/xlnet-Pytorch","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/https-seyhan/BugAI","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindCode-4/code-5/tree/main/xlnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/anli","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pauldevos/python-notes","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jonahwinninghoff/Text-Summarization","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pwc-1/Paper-9/tree/main/5/xlnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pwc-1/Paper-9/tree/main/1/xlnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lvyufeng/bert4ms/blob/master/bert4ms/models/xlnet.py","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/zihangdai/xlnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cuhksz-nlp/SAPar","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/listenviolet/XLNet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/joshuaWang-bit/Textclassification-pytorch","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chesterdu/contrastive_summary","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/samwisegamjeee/pytorch-transformers","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/PaddlePaddle/PaddleNLP/tree/develop/examples/language_model/xlnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MS-P3/code7/tree/main/xlnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kaushaltrivedi/fast-bert","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zaradana/Fast_BERT","reach":null}],"summary":{"ran_draft_wrong":5,"ran":3,"ran_honours":2,"unverified":14},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":22,"ran":9,"repositories":5}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"0f786c407fb1ee4c","entry":"swish","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_bert.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_bert.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0f786c407fb1ee4c"}},{"code_sha256_prefix":"624bba127433de69","entry":"SupConLoss","repo":"chesterdu/contrastive_summary","repo_kind":"listed","path":"src/model.py","file_url":"https://github.com/chesterdu/contrastive_summary/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"624bba127433de69"}},{"code_sha256_prefix":"30647553d552c86f","entry":"XLNet","repo":"fanchenyou/transformer-study","repo_kind":"listed","path":"transformer_xlnet_4.py","file_url":"https://github.com/fanchenyou/transformer-study/blob/HEAD/transformer_xlnet_4.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"30647553d552c86f"}},{"code_sha256_prefix":"35ef111fc00c21b4","entry":"build_tf_to_pytorch_map","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_transfo_xl.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_transfo_xl.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"35ef111fc00c21b4"}},{"code_sha256_prefix":"8d23fbe2b99b840b","entry":"gelu","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_gpt2.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_gpt2.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8d23fbe2b99b840b"}},{"code_sha256_prefix":"fdc64f4c72036ae4","entry":"gelu","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_bert.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_bert.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fdc64f4c72036ae4"}},{"code_sha256_prefix":"d30d6c3098df2c42","entry":"prune_linear_layer","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_utils.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d30d6c3098df2c42"}},{"code_sha256_prefix":"4276ef7411886c72","entry":"relative_positional_encoding","repo":"zihangdai/xlnet","repo_kind":"official","path":"modeling.py","file_url":"https://github.com/zihangdai/xlnet/blob/HEAD/modeling.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4276ef7411886c72"}},{"code_sha256_prefix":"93228a3eb5c4d179","entry":"sample_logits","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_transfo_xl_utilities.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_transfo_xl_utilities.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"93228a3eb5c4d179"}},{"code_sha256_prefix":"1370bb66bcbac8f1","entry":"scl_model_Xlnet","repo":"chesterdu/contrastive_summary","repo_kind":"listed","path":"src/model.py","file_url":"https://github.com/chesterdu/contrastive_summary/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1370bb66bcbac8f1"}},{"code_sha256_prefix":"ea4abb20faaddb4d","entry":"XLNetForMultiLabelSequenceClassification","repo":"zaradana/Fast_BERT","repo_kind":"listed","path":"0_fast_bert/modeling.py","file_url":"https://github.com/zaradana/Fast_BERT/blob/HEAD/0_fast_bert/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ea4abb20faaddb4d"}},{"code_sha256_prefix":"de032896e6064f69","entry":"XLNetForMultiLabelSequenceClassification","repo":"utterworks/fast-bert","repo_kind":"listed","path":"fast_bert/modeling.py","file_url":"https://github.com/utterworks/fast-bert/blob/HEAD/fast_bert/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de032896e6064f69"}},{"code_sha256_prefix":"c92ba449e48ecffe","entry":"build_tf_xlnet_to_pytorch_map","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_xlnet.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_xlnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c92ba449e48ecffe"}},{"code_sha256_prefix":"b05b2bb38c0e4e72","entry":"gelu","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_xlnet.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_xlnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b05b2bb38c0e4e72"}},{"code_sha256_prefix":"b3c2618f6b7fb60c","entry":"gelu","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_xlm.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_xlm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b3c2618f6b7fb60c"}},{"code_sha256_prefix":"f5d8d44f65a08b76","entry":"get_masks","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_xlm.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_xlm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f5d8d44f65a08b76"}},{"code_sha256_prefix":"03a94dbb436249a3","entry":"load_tf_weights_in_bert","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_bert.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_bert.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03a94dbb436249a3"}},{"code_sha256_prefix":"07bb0220eb5b609f","entry":"load_tf_weights_in_gpt2","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_gpt2.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_gpt2.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"07bb0220eb5b609f"}},{"code_sha256_prefix":"42ee168c3ae12e60","entry":"load_tf_weights_in_openai_gpt","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_openai.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_openai.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"42ee168c3ae12e60"}},{"code_sha256_prefix":"23d246e59ef9447b","entry":"load_tf_weights_in_transfo_xl","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_transfo_xl.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_transfo_xl.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"23d246e59ef9447b"}},{"code_sha256_prefix":"4a94662ff3b5c5cf","entry":"load_tf_weights_in_xlnet","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_xlnet.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_xlnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a94662ff3b5c5cf"}},{"code_sha256_prefix":"7183a17dd79327a8","entry":"prune_conv1d_layer","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_utils.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7183a17dd79327a8"}},{"code_sha256_prefix":"e31093ffc8275cff","entry":"prune_layer","repo":"samwisegamjeee/pytorch-transformers","repo_kind":"listed","path":"pytorch_transformers/modeling_utils.py","file_url":"https://github.com/samwisegamjeee/pytorch-transformers/blob/HEAD/pytorch_transformers/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e31093ffc8275cff"}},{"code_sha256_prefix":"346015e252bc02c1","entry":"transformer_xl","repo":"zihangdai/xlnet","repo_kind":"official","path":"modeling.py","file_url":"https://github.com/zihangdai/xlnet/blob/HEAD/modeling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"346015e252bc02c1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}