{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/well-read-students-learn-better-the-impact-of","title":"Well-Read Students Learn Better: On the Importance of Pre-training Compact Models","arxiv_id":"1908.08962","date":"2019-08-23","proceeding":"ICLR 2020 1","authors":["Iulia Turc","Ming-Wei Chang","Kenton Lee","Kristina Toutanova"],"abstract":"Recent developments in natural language representations have been accompanied by large and expensive models that leverage vast amounts of general-domain text through self-supervised pre-training. Due to the cost of applying such models to down-stream tasks, several model compression techniques on pre-trained language representations have been proposed (Sun et al., 2019; Sanh, 2019). However, surprisingly, the simple baseline of just pre-training and fine-tuning compact models has been overlooked. In this paper, we first show that pre-training remains important in the context of smaller architectures, and fine-tuning pre-trained compact models can be competitive to more elaborate methods proposed in concurrent work. Starting with pre-trained compact models, we then explore transferring task knowledge from large fine-tuned models through standard knowledge distillation. The resulting simple, yet effective and general algorithm, Pre-trained Distillation, brings further improvements. Through extensive experiments, we more generally explore the interaction between pre-training and distillation under two variables that have been under-studied: model size and properties of unlabeled task data. One surprising observation is that they have a compound effect even when sequentially applied on the same data. To accelerate future research, we will make our 24 pre-trained miniature BERT models publicly available.","url_abs":"https://arxiv.org/abs/1908.08962v2","url_pdf":"https://arxiv.org/pdf/1908.08962v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/google-research/bert","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/Arthurizijar/Bert_Airport","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/Caesarzhang/bert-zh","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/Chonwai/Learning_BERT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/DeligientSloth/QQsim","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/DeligientSloth/bert-tensorflow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/MOHAMEDELDAKDOUKY/bert_adjusted","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/Maz101/Bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/Maz101/pretrainingdataset","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/PAIR-code/lit","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/RenXiangyuan/tf_bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/SpikeKing/My-Bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/StoneGH/bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/TonyX19/bert_hyperpartisan","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/Walter-B/bert-20-classes","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/XINZXZHANG/Bert_repo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/abhiagg92/bert_sentiment","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/coronazap/bert_client","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/cospplay/bert-master","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/cuber2460/bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/dzqjorking/transpose","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/geondopark/ckd","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/goodluck4s/bert-","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/google-research/bleurt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/google-research/tapas","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/halo090770/bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/krantirk/BERT-Pretrained-model","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/llx666/git_demo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/paolanu/BERT_epitope","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/pingheng001/Cnn-Bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/ricardordb/bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/rohan400/PepperQA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/somiltg/bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/the-Quert/iNLPfun","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/theQuert/inlpfun","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/thecodemasterk/BERT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/thousandvoices/ok_ml_cup","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/xiaopp123/bert_explain","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/xitianxiaofeixue/BERT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"well-read-students-learn-better-the-impact-of","repo_url":"https://github.com/yuhangT/tf_bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1908.08962","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1908.08962"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dzqjorking/transpose","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/tapas","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TonyX19/bert_hyperpartisan","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DeligientSloth/QQsim","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/PAIR-code/lit","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MOHAMEDELDAKDOUKY/bert_adjusted","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cospplay/bert-master","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Arthurizijar/Bert_Airport","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/paolanu/BERT_epitope","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SpikeKing/My-Bert","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/goodluck4s/bert-","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/StoneGH/bert","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Caesarzhang/bert-zh","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cuber2460/bert","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DeligientSloth/bert-tensorflow","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/halo090770/bert","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/RenXiangyuan/tf_bert","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thecodemasterk/BERT","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/llx666/git_demo","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/somiltg/bert","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Chonwai/Learning_BERT","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/abhiagg92/bert_sentiment","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/coronazap/bert_client","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xiaopp123/bert_explain","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ricardordb/bert","reach":null}],"summary":{"ran_draft_wrong":5,"ran_honours":1,"unverified":26},"by_repo_kind":{"listed":{"samples":32,"ran":6,"repositories":12}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1923fc05163d207d","entry":"convert_to_unicode","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"tokenization.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/tokenization.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1923fc05163d207d"}},{"code_sha256_prefix":"d08f324f950148de","entry":"gelu","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"modeling.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/modeling.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d08f324f950148de"}},{"code_sha256_prefix":"5842ee67106321d6","entry":"get_activation","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"modeling.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/modeling.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5842ee67106321d6"}},{"code_sha256_prefix":"aa6bc32d9c17a2f1","entry":"get_tokens","repo":"somiltg/bert","repo_kind":"listed","path":"evaluate-v2.0.py","file_url":"https://github.com/somiltg/bert/blob/HEAD/evaluate-v2.0.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa6bc32d9c17a2f1"}},{"code_sha256_prefix":"e91c44f5680827e2","entry":"make_qid_to_has_ans","repo":"somiltg/bert","repo_kind":"listed","path":"evaluate-v2.0.py","file_url":"https://github.com/somiltg/bert/blob/HEAD/evaluate-v2.0.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e91c44f5680827e2"}},{"code_sha256_prefix":"6a96435eba311b08","entry":"normalize_answer","repo":"somiltg/bert","repo_kind":"listed","path":"evaluate-v2.0.py","file_url":"https://github.com/somiltg/bert/blob/HEAD/evaluate-v2.0.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6a96435eba311b08"}},{"code_sha256_prefix":"7ba03346caea013d","entry":"clones","repo":"the-Quert/iNLPfun","repo_kind":"listed","path":"custom/encoder_ref.py","file_url":"https://github.com/the-Quert/iNLPfun/blob/HEAD/custom/encoder_ref.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7ba03346caea013d"}},{"code_sha256_prefix":"b4a25fd2a133d044","entry":"convert_example_to_feature","repo":"paolanu/BERT_epitope","repo_kind":"listed","path":"convert_examples_to_features.py","file_url":"https://github.com/paolanu/BERT_epitope/blob/HEAD/convert_examples_to_features.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b4a25fd2a133d044"}},{"code_sha256_prefix":"b24b3b9c700da839","entry":"create_float_feature","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"create_pretraining_data.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/create_pretraining_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b24b3b9c700da839"}},{"code_sha256_prefix":"dd3ef48a15aec0cd","entry":"create_int_feature","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"create_pretraining_data.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/create_pretraining_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dd3ef48a15aec0cd"}},{"code_sha256_prefix":"c070006ff55cd83a","entry":"create_optimizer","repo":"MOHAMEDELDAKDOUKY/bert_adjusted","repo_kind":"listed","path":"optimization.py","file_url":"https://github.com/MOHAMEDELDAKDOUKY/bert_adjusted/blob/HEAD/optimization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c070006ff55cd83a"}},{"code_sha256_prefix":"021cbaa59ced6dac","entry":"create_optimizer","repo":"SpikeKing/My-Bert","repo_kind":"listed","path":"optimization.py","file_url":"https://github.com/SpikeKing/My-Bert/blob/HEAD/optimization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"021cbaa59ced6dac"}},{"code_sha256_prefix":"5f1c73e253f08c93","entry":"create_optimizer","repo":"halo090770/bert","repo_kind":"listed","path":"optimization_1.py","file_url":"https://github.com/halo090770/bert/blob/HEAD/optimization_1.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5f1c73e253f08c93"}},{"code_sha256_prefix":"85813dd1e897d90f","entry":"download_and_load_datasets","repo":"Chonwai/Learning_BERT","repo_kind":"listed","path":"predicting_movie_reviews_with_bert_on_tf_hub.py","file_url":"https://github.com/Chonwai/Learning_BERT/blob/HEAD/predicting_movie_reviews_with_bert_on_tf_hub.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85813dd1e897d90f"}},{"code_sha256_prefix":"06627935dcb8a0fe","entry":"extractLabelXml","repo":"TonyX19/bert_hyperpartisan","repo_kind":"listed","path":"seperate.py","file_url":"https://github.com/TonyX19/bert_hyperpartisan/blob/HEAD/seperate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"06627935dcb8a0fe"}},{"code_sha256_prefix":"1b46c9ce24fc6db3","entry":"file_based_input_fn_builder","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"run_classifier.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/run_classifier.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1b46c9ce24fc6db3"}},{"code_sha256_prefix":"28b353970e44ff54","entry":"file_based_input_fn_builder","repo":"Maz101/Bert","repo_kind":"listed","path":"run_classifier.py","file_url":"https://github.com/Maz101/Bert/blob/HEAD/run_classifier.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"28b353970e44ff54"}},{"code_sha256_prefix":"50958618b65e514e","entry":"get_assignment_map_from_checkpoint","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"modeling.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50958618b65e514e"}},{"code_sha256_prefix":"c54b935b72798660","entry":"get_tasks","repo":"the-Quert/iNLPfun","repo_kind":"listed","path":"code/download_glue_data.py","file_url":"https://github.com/the-Quert/iNLPfun/blob/HEAD/code/download_glue_data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c54b935b72798660"}},{"code_sha256_prefix":"5edf8fc5e1137648","entry":"init_predict_var","repo":"StoneGH/bert","repo_kind":"listed","path":"freeze_graph.py","file_url":"https://github.com/StoneGH/bert/blob/HEAD/freeze_graph.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5edf8fc5e1137648"}},{"code_sha256_prefix":"47a3c792cebb6603","entry":"input_fn_builder","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"extract_features.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/extract_features.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"47a3c792cebb6603"}},{"code_sha256_prefix":"ce566df49585f2b5","entry":"loadText","repo":"TonyX19/bert_hyperpartisan","repo_kind":"listed","path":"seperate.py","file_url":"https://github.com/TonyX19/bert_hyperpartisan/blob/HEAD/seperate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce566df49585f2b5"}},{"code_sha256_prefix":"a181d94c3334306d","entry":"load_dataset","repo":"Chonwai/Learning_BERT","repo_kind":"listed","path":"predicting_movie_reviews_with_bert_on_tf_hub.py","file_url":"https://github.com/Chonwai/Learning_BERT/blob/HEAD/predicting_movie_reviews_with_bert_on_tf_hub.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a181d94c3334306d"}},{"code_sha256_prefix":"0ccbf8e553bab00d","entry":"load_directory_data","repo":"Chonwai/Learning_BERT","repo_kind":"listed","path":"predicting_movie_reviews_with_bert_on_tf_hub.py","file_url":"https://github.com/Chonwai/Learning_BERT/blob/HEAD/predicting_movie_reviews_with_bert_on_tf_hub.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ccbf8e553bab00d"}},{"code_sha256_prefix":"ff83ccc8b0b6462d","entry":"load_vocab","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"tokenization.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/tokenization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff83ccc8b0b6462d"}},{"code_sha256_prefix":"30b05f2a415dd132","entry":"load_vocab","repo":"Maz101/Bert","repo_kind":"listed","path":"tokenization.py","file_url":"https://github.com/Maz101/Bert/blob/HEAD/tokenization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"30b05f2a415dd132"}},{"code_sha256_prefix":"21c6f1a7ac31a2bd","entry":"load_vocab","repo":"coronazap/bert_client","repo_kind":"listed","path":"tokenization.py","file_url":"https://github.com/coronazap/bert_client/blob/HEAD/tokenization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"21c6f1a7ac31a2bd"}},{"code_sha256_prefix":"0e5615f8994003cf","entry":"printable_text","repo":"Arthurizijar/Bert_Airport","repo_kind":"listed","path":"tokenization.py","file_url":"https://github.com/Arthurizijar/Bert_Airport/blob/HEAD/tokenization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0e5615f8994003cf"}},{"code_sha256_prefix":"7b79d58035cd12c2","entry":"process_output","repo":"coronazap/bert_client","repo_kind":"listed","path":"run_squad.py","file_url":"https://github.com/coronazap/bert_client/blob/HEAD/run_squad.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b79d58035cd12c2"}},{"code_sha256_prefix":"b57e2fc59104beb6","entry":"process_result","repo":"coronazap/bert_client","repo_kind":"listed","path":"run_squad.py","file_url":"https://github.com/coronazap/bert_client/blob/HEAD/run_squad.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b57e2fc59104beb6"}},{"code_sha256_prefix":"3cbc0c492348e83d","entry":"read_squad_examples","repo":"coronazap/bert_client","repo_kind":"listed","path":"run_squad.py","file_url":"https://github.com/coronazap/bert_client/blob/HEAD/run_squad.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3cbc0c492348e83d"}},{"code_sha256_prefix":"9b7c0abd85734790","entry":"set_logger","repo":"StoneGH/bert","repo_kind":"listed","path":"freeze_graph.py","file_url":"https://github.com/StoneGH/bert/blob/HEAD/freeze_graph.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9b7c0abd85734790"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}