{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unified-low-resource-sequence-labeling-by","title":"Unified Low-Resource Sequence Labeling by Sample-Aware Dynamic Sparse Finetuning","arxiv_id":"2311.03748","date":"2023-11-07","proceeding":null,"authors":["Sarkar Snigdha Sarathi Das","Ranran Haoran Zhang","Peng Shi","Wenpeng Yin","Rui Zhang"],"abstract":"Unified Sequence Labeling that articulates different sequence labeling problems such as Named Entity Recognition, Relation Extraction, Semantic Role Labeling, etc. in a generalized sequence-to-sequence format opens up the opportunity to make the maximum utilization of large language model knowledge toward structured prediction. Unfortunately, this requires formatting them into specialized augmented format unknown to the base pretrained language model (PLMs) necessitating finetuning to the target format. This significantly bounds its usefulness in data-limited settings where finetuning large models cannot properly generalize to the target format. To address this challenge and leverage PLM knowledge effectively, we propose FISH-DIP, a sample-aware dynamic sparse finetuning strategy that selectively focuses on a fraction of parameters, informed by feedback from highly regressing examples, during the fine-tuning process. By leveraging the dynamism of sparsity, our approach mitigates the impact of well-learned samples and prioritizes underperforming instances for improvement in generalization. Across five tasks of sequence labeling, we demonstrate that FISH-DIP can smoothly optimize the model in low resource settings offering upto 40% performance improvements over full fine-tuning depending on target evaluation settings. Also, compared to in-context learning and other parameter-efficient fine-tuning approaches, FISH-DIP performs comparably or better, notably in extreme low-resource settings.","url_abs":"https://arxiv.org/abs/2311.03748v1","url_pdf":"https://arxiv.org/pdf/2311.03748v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unified-low-resource-sequence-labeling-by","repo_url":"https://github.com/psunlpgroup/fish-dip","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"relation-extraction","task_name":"Relation Extraction"},{"task_slug":"semantic-role-labeling","task_name":"Semantic Role Labeling"},{"task_slug":"structured-prediction","task_name":"Structured Prediction"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.03748","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.03748"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/psunlpgroup/fish-dip","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":10,"unverified":4},"by_repo_kind":{"official":{"samples":14,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"84d12291a8245407","entry":"augment_sentence","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/utils.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"84d12291a8245407"}},{"code_sha256_prefix":"2d71e2917845b6ac","entry":"b_cubed","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/coreference_metrics.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/coreference_metrics.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d71e2917845b6ac"}},{"code_sha256_prefix":"9055c123057f71e4","entry":"calculate_gradients","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"fish_dip.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/fish_dip.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9055c123057f71e4"}},{"code_sha256_prefix":"4906e8d0bdd256bf","entry":"expand_tokens","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/utils.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4906e8d0bdd256bf"}},{"code_sha256_prefix":"3e9c0f90d3b4010c","entry":"get_avg_results","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/evaluate.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3e9c0f90d3b4010c"}},{"code_sha256_prefix":"3ef809361ff6c485","entry":"get_episode_indices","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/utils.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3ef809361ff6c485"}},{"code_sha256_prefix":"d129d44186c9e033","entry":"get_slot_information","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"preprocess_multiwoz/extract_examples.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/preprocess_multiwoz/extract_examples.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d129d44186c9e033"}},{"code_sha256_prefix":"404087a3ff94d6e0","entry":"register_dataset","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/datasets_all.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/datasets_all.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"404087a3ff94d6e0"}},{"code_sha256_prefix":"4afe64752662fe3b","entry":"register_input_format","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/input_formats.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/input_formats.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4afe64752662fe3b"}},{"code_sha256_prefix":"ba71a83cd3836e77","entry":"register_output_format","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/output_formats.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/output_formats.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba71a83cd3836e77"}},{"code_sha256_prefix":"614af423a25ed096","entry":"create_mask_gradient","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"fish_dip.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/fish_dip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"614af423a25ed096"}},{"code_sha256_prefix":"44ea1242eeeb38ac","entry":"evaluate_documents","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/coreference_metrics.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/coreference_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"44ea1242eeeb38ac"}},{"code_sha256_prefix":"5dc251a1bb16c94f","entry":"f1","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"augment/coreference_metrics.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/augment/coreference_metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5dc251a1bb16c94f"}},{"code_sha256_prefix":"c6855e904db1e7ca","entry":"read_file","repo":"psunlpgroup/fish-dip","repo_kind":"official","path":"preprocess_multiwoz/extract_examples.py","file_url":"https://github.com/psunlpgroup/fish-dip/blob/HEAD/preprocess_multiwoz/extract_examples.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c6855e904db1e7ca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}