{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/zen-pre-training-chinese-text-encoder","title":"ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations","arxiv_id":"1911.00720","date":"2019-11-02","proceeding":"Findings of the Association for Computational Linguistics 2020","authors":["Shizhe Diao","Jiaxin Bai","Yan Song","Tong Zhang","Yonggang Wang"],"abstract":"The pre-training of text encoders normally processes text as a sequence of tokens corresponding to small text units, such as word pieces in English and characters in Chinese. It omits information carried by larger text granularity, and thus the encoders cannot easily adapt to certain combinations of characters. This leads to a loss of important semantic information, which is especially problematic for Chinese because the language does not have explicit word boundaries. In this paper, we propose ZEN, a BERT-based Chinese (Z) text encoder Enhanced by N-gram representations, where different combinations of characters are considered during training. As a result, potential word or phase boundaries are explicitly pre-trained and fine-tuned with the character encoder (BERT). Therefore ZEN incorporates the comprehensive information of both the character sequence and words or phrases it contains. Experimental results illustrated the effectiveness of ZEN on a series of Chinese NLP tasks. We show that ZEN, using less resource than other published encoders, can achieve state-of-the-art performance on most tasks. Moreover, it is shown that reasonable performance can be obtained when ZEN is trained on a small corpus, which is important for applying pre-training techniques to scenarios with limited data. The code and pre-trained models of ZEN are available at https://github.com/sinovation/zen.","url_abs":"https://arxiv.org/abs/1911.00720v1","url_pdf":"https://arxiv.org/pdf/1911.00720v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zen-pre-training-chinese-text-encoder","repo_url":"https://github.com/sinovation/ZEN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"zen-pre-training-chinese-text-encoder","repo_url":"https://github.com/SVAIGBA/TwASP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"zen-pre-training-chinese-text-encoder","repo_url":"https://github.com/SVAIGBA/WMSeg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"zen-pre-training-chinese-text-encoder","repo_url":"https://github.com/YYGe01/ZEN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"paper_slug":"zen-pre-training-chinese-text-encoder","repo_url":"https://github.com/cuhksz-nlp/SAPar","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"zen-pre-training-chinese-text-encoder","repo_url":"https://github.com/cuhksz-nlp/het-mc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"zen-pre-training-chinese-text-encoder","repo_url":"https://github.com/cuhksz-nlp/mcasp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"chinese-named-entity-recognition","task_name":"Chinese Named Entity Recognition"},{"task_slug":"chinese-word-segmentation","task_name":"Chinese Word Segmentation"},{"task_slug":"document-classification","task_name":"Document Classification"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"part-of-speech-tagging","task_name":"Part-Of-Speech Tagging"},{"task_slug":"sentence-pair-modeling","task_name":"Sentence Pair Modeling"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/chinese-named-entity-recognition-on-msra","task":"Chinese Named Entity Recognition","dataset":"MSRA","model":"ZEN (Init with Chinese BERT)","rank_in_archive_order":9,"of":21,"metrics":{"F1":"95.25"},"uses_additional_data":false},{"leaderboard":"/sota/chinese-named-entity-recognition-on-msra","task":"Chinese Named Entity Recognition","dataset":"MSRA","model":"ZEN (Random Init)","rank_in_archive_order":18,"of":21,"metrics":{"F1":"93.24"},"uses_additional_data":false},{"leaderboard":"/sota/chinese-word-segmentation-on-msr","task":"Chinese Word Segmentation","dataset":"MSR","model":"ZEN (Init with Chinese BERT)","rank_in_archive_order":4,"of":6,"metrics":{"F1":"98.35"},"uses_additional_data":false},{"leaderboard":"/sota/chinese-word-segmentation-on-msr","task":"Chinese Word Segmentation","dataset":"MSR","model":"ZEN (Random Init)","rank_in_archive_order":6,"of":6,"metrics":{"F1":"97.89"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1911.00720","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1911.00720"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SVAIGBA/WMSeg","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sinovation/ZEN","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/YYGe01/ZEN","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cuhksz-nlp/SAPar","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cuhksz-nlp/mcasp","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SVAIGBA/TwASP","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cuhksz-nlp/het-mc","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":6,"ran_honours":1,"unverified":28},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":33,"ran":6,"repositories":4}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0f786c407fb1ee4c","entry":"swish","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling_openai.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling_openai.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f786c407fb1ee4c"}},{"code_sha256_prefix":"35ef111fc00c21b4","entry":"build_tf_to_pytorch_map","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling_transfo_xl.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling_transfo_xl.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35ef111fc00c21b4"}},{"code_sha256_prefix":"ba2b25faa32a1249","entry":"cws_evaluate_word_PRF","repo":"sinovation/ZEN","repo_kind":"official","path":"examples/run_token_level_classification.py","file_url":"https://github.com/sinovation/ZEN/blob/HEAD/examples/run_token_level_classification.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba2b25faa32a1249"}},{"code_sha256_prefix":"8d23fbe2b99b840b","entry":"gelu","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling_gpt2.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling_gpt2.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d23fbe2b99b840b"}},{"code_sha256_prefix":"fdc64f4c72036ae4","entry":"gelu","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fdc64f4c72036ae4"}},{"code_sha256_prefix":"d30d6c3098df2c42","entry":"prune_linear_layer","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d30d6c3098df2c42"}},{"code_sha256_prefix":"93228a3eb5c4d179","entry":"sample_logits","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling_transfo_xl_utilities.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling_transfo_xl_utilities.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93228a3eb5c4d179"}},{"code_sha256_prefix":"000595907a687ec4","entry":"convert_example_to_features","repo":"sinovation/ZEN","repo_kind":"official","path":"examples/run_pre_train.py","file_url":"https://github.com/sinovation/ZEN/blob/HEAD/examples/run_pre_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"000595907a687ec4"}},{"code_sha256_prefix":"4892acce036705d1","entry":"count_n_gram","repo":"SVAIGBA/WMSeg","repo_kind":"listed","path":"wmseg_helper.py","file_url":"https://github.com/SVAIGBA/WMSeg/blob/HEAD/wmseg_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4892acce036705d1"}},{"code_sha256_prefix":"1a310e8f6b36323b","entry":"cws_evaluate_OOV","repo":"SVAIGBA/WMSeg","repo_kind":"listed","path":"wmseg_eval.py","file_url":"https://github.com/SVAIGBA/WMSeg/blob/HEAD/wmseg_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a310e8f6b36323b"}},{"code_sha256_prefix":"0421eab98cc3b9a9","entry":"cws_evaluate_word_PRF","repo":"SVAIGBA/WMSeg","repo_kind":"listed","path":"wmseg_eval.py","file_url":"https://github.com/SVAIGBA/WMSeg/blob/HEAD/wmseg_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0421eab98cc3b9a9"}},{"code_sha256_prefix":"301c6b5f8def3f58","entry":"dlg","repo":"SVAIGBA/WMSeg","repo_kind":"listed","path":"wmseg_helper.py","file_url":"https://github.com/SVAIGBA/WMSeg/blob/HEAD/wmseg_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"301c6b5f8def3f58"}},{"code_sha256_prefix":"34ec071d80ec7278","entry":"dlg","repo":"cuhksz-nlp/mcasp","repo_kind":"listed","path":"mcasp_helper.py","file_url":"https://github.com/cuhksz-nlp/mcasp/blob/HEAD/mcasp_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34ec071d80ec7278"}},{"code_sha256_prefix":"87867d95beeee94a","entry":"eval_sentence","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"twasp_eval.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/twasp_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87867d95beeee94a"}},{"code_sha256_prefix":"fad09e6582176e41","entry":"eval_sentence","repo":"SVAIGBA/WMSeg","repo_kind":"listed","path":"wmseg_eval.py","file_url":"https://github.com/SVAIGBA/WMSeg/blob/HEAD/wmseg_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fad09e6582176e41"}},{"code_sha256_prefix":"116ccd8fe121ef8f","entry":"eval_sentence","repo":"cuhksz-nlp/mcasp","repo_kind":"listed","path":"mcasp_eval.py","file_url":"https://github.com/cuhksz-nlp/mcasp/blob/HEAD/mcasp_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"116ccd8fe121ef8f"}},{"code_sha256_prefix":"99b87b401d1fae1e","entry":"get_vocab","repo":"cuhksz-nlp/het-mc","repo_kind":"listed","path":"hetmc_helper.py","file_url":"https://github.com/cuhksz-nlp/het-mc/blob/HEAD/hetmc_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99b87b401d1fae1e"}},{"code_sha256_prefix":"98046cec1e6090e5","entry":"get_word2id","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"twasp_helper.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/twasp_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98046cec1e6090e5"}},{"code_sha256_prefix":"ecd45f0d33c0aff5","entry":"get_word2id","repo":"cuhksz-nlp/mcasp","repo_kind":"listed","path":"mcasp_helper.py","file_url":"https://github.com/cuhksz-nlp/mcasp/blob/HEAD/mcasp_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ecd45f0d33c0aff5"}},{"code_sha256_prefix":"3872690ce9ded1c4","entry":"load_json","repo":"cuhksz-nlp/het-mc","repo_kind":"listed","path":"hetmc_helper.py","file_url":"https://github.com/cuhksz-nlp/het-mc/blob/HEAD/hetmc_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3872690ce9ded1c4"}},{"code_sha256_prefix":"b1a7180868dd8d6a","entry":"load_tf_weights_in_bert","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1a7180868dd8d6a"}},{"code_sha256_prefix":"a62604f618a8cc82","entry":"load_tf_weights_in_gpt2","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling_gpt2.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling_gpt2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a62604f618a8cc82"}},{"code_sha256_prefix":"46f305675277e757","entry":"load_tf_weights_in_openai_gpt","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling_openai.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling_openai.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"46f305675277e757"}},{"code_sha256_prefix":"1ebc2bf2c3cddc4d","entry":"load_tf_weights_in_transfo_xl","repo":"cuhksz-nlp/het-mc","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling_transfo_xl.py","file_url":"https://github.com/cuhksz-nlp/het-mc/blob/HEAD/pytorch_pretrained_bert/modeling_transfo_xl.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ebc2bf2c3cddc4d"}},{"code_sha256_prefix":"342218755bdf47ed","entry":"pos_evaluate_OOV","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"twasp_eval.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/twasp_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"342218755bdf47ed"}},{"code_sha256_prefix":"3f4a9392fcb8695c","entry":"pos_evaluate_OOV","repo":"cuhksz-nlp/mcasp","repo_kind":"listed","path":"mcasp_eval.py","file_url":"https://github.com/cuhksz-nlp/mcasp/blob/HEAD/mcasp_eval.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3f4a9392fcb8695c"}},{"code_sha256_prefix":"f255dd42f46c11c1","entry":"prune_conv1d_layer","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"pytorch_pretrained_bert/modeling_gpt2.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/pytorch_pretrained_bert/modeling_gpt2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f255dd42f46c11c1"}},{"code_sha256_prefix":"c9622c7a1d0912d6","entry":"read_dialog","repo":"cuhksz-nlp/het-mc","repo_kind":"listed","path":"hetmc_helper.py","file_url":"https://github.com/cuhksz-nlp/het-mc/blob/HEAD/hetmc_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c9622c7a1d0912d6"}},{"code_sha256_prefix":"5b7c711347a2c214","entry":"read_sentence","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"twasp_model.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/twasp_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b7c711347a2c214"}},{"code_sha256_prefix":"4d322567015518e5","entry":"read_sentence","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"twasp_helper.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/twasp_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d322567015518e5"}},{"code_sha256_prefix":"352e22ef30c1f7b7","entry":"read_tsv","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"twasp_helper.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/twasp_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"352e22ef30c1f7b7"}},{"code_sha256_prefix":"efebd15616e8be5b","entry":"read_tsv","repo":"SVAIGBA/WMSeg","repo_kind":"listed","path":"wmseg_helper.py","file_url":"https://github.com/SVAIGBA/WMSeg/blob/HEAD/wmseg_helper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"efebd15616e8be5b"}},{"code_sha256_prefix":"013a1a80d4e2be45","entry":"readfile","repo":"SVAIGBA/TwASP","repo_kind":"listed","path":"twasp_model.py","file_url":"https://github.com/SVAIGBA/TwASP/blob/HEAD/twasp_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"013a1a80d4e2be45"}},{"code_sha256_prefix":"3e642a8045e6ba92","entry":"readfile","repo":"SVAIGBA/WMSeg","repo_kind":"listed","path":"wmseg_model.py","file_url":"https://github.com/SVAIGBA/WMSeg/blob/HEAD/wmseg_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e642a8045e6ba92"}},{"code_sha256_prefix":"9b29b0de8fad09f9","entry":"readsentence","repo":"SVAIGBA/WMSeg","repo_kind":"listed","path":"wmseg_model.py","file_url":"https://github.com/SVAIGBA/WMSeg/blob/HEAD/wmseg_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b29b0de8fad09f9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}