{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-simple-hash-based-early-exiting-approach-1","title":"A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation","arxiv_id":"2203.01670","date":"2022-03-03","proceeding":"Findings (ACL) 2022 5","authors":["Tianxiang Sun","Xiangyang Liu","Wei Zhu","Zhichao Geng","Lingling Wu","Yilong He","Yuan Ni","Guotong Xie","Xuanjing Huang","Xipeng Qiu"],"abstract":"Early exiting allows instances to exit at different layers according to the estimation of difficulty. Previous works usually adopt heuristic metrics such as the entropy of internal outputs to measure instance difficulty, which suffers from generalization and threshold-tuning. In contrast, learning to exit, or learning to predict instance difficulty is a more appealing way. Though some effort has been devoted to employing such \"learn-to-exit\" modules, it is still unknown whether and how well the instance difficulty can be learned. As a response, we first conduct experiments on the learnability of instance difficulty, which demonstrates that modern neural models perform poorly on predicting instance difficulty. Based on this observation, we propose a simple-yet-effective Hash-based Early Exiting approach (HashEE) that replaces the learn-to-exit modules with hash functions to assign each token to a fixed exiting layer. Different from previous methods, HashEE requires no internal classifiers nor extra parameters, and therefore is more efficient. Experimental results on classification, regression, and generation tasks demonstrate that HashEE can achieve higher performance with fewer FLOPs and inference time compared with previous state-of-the-art early exiting methods.","url_abs":"https://arxiv.org/abs/2203.01670v1","url_pdf":"https://arxiv.org/pdf/2203.01670v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-simple-hash-based-early-exiting-approach-1","repo_url":"https://github.com/txsun1997/hashee","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"early-exiting","method_name":"Early exiting"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2203.01670","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.01670"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/txsun1997/hashee","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ce106170b591f393","entry":"attention_mask_func","repo":"txsun1997/hashee","repo_kind":"official","path":"models/modeling_elasticbert_approx_hashee.py","file_url":"https://github.com/txsun1997/hashee/blob/HEAD/models/modeling_elasticbert_approx_hashee.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce106170b591f393"}},{"code_sha256_prefix":"eaf0d81840f1d648","entry":"get_dropped_hidden","repo":"txsun1997/hashee","repo_kind":"official","path":"models/modeling_elasticbert_hashee.py","file_url":"https://github.com/txsun1997/hashee/blob/HEAD/models/modeling_elasticbert_hashee.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eaf0d81840f1d648"}},{"code_sha256_prefix":"89bec46d4c56090c","entry":"get_extended_attention_mask","repo":"txsun1997/hashee","repo_kind":"official","path":"models/modeling_elasticbert_approx_hashee.py","file_url":"https://github.com/txsun1997/hashee/blob/HEAD/models/modeling_elasticbert_approx_hashee.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89bec46d4c56090c"}},{"code_sha256_prefix":"4977b7509775eb23","entry":"get_reduced_hidden","repo":"txsun1997/hashee","repo_kind":"official","path":"models/modeling_elasticbert_hashee.py","file_url":"https://github.com/txsun1997/hashee/blob/HEAD/models/modeling_elasticbert_hashee.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4977b7509775eb23"}},{"code_sha256_prefix":"2cff6510499768ce","entry":"get_start_piece_outputs","repo":"txsun1997/hashee","repo_kind":"official","path":"models/modeling_elasticbert_approx_hashee.py","file_url":"https://github.com/txsun1997/hashee/blob/HEAD/models/modeling_elasticbert_approx_hashee.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2cff6510499768ce"}},{"code_sha256_prefix":"26be70dca3249c0b","entry":"load_tf_weights_in_bert","repo":"txsun1997/hashee","repo_kind":"official","path":"difficulty/modeling_bert.py","file_url":"https://github.com/txsun1997/hashee/blob/HEAD/difficulty/modeling_bert.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26be70dca3249c0b"}},{"code_sha256_prefix":"a834d201022a2476","entry":"shift_nonzero","repo":"txsun1997/hashee","repo_kind":"official","path":"models/modeling_elasticbert_hashee.py","file_url":"https://github.com/txsun1997/hashee/blob/HEAD/models/modeling_elasticbert_hashee.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a834d201022a2476"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}