{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fingerprinting-fine-tuned-language-models-in","title":"Fingerprinting Fine-tuned Language Models in the Wild","arxiv_id":"2106.01703","date":"2021-06-03","proceeding":"Findings (ACL) 2021 8","authors":["Nirav Diwan","Tanmoy Chakravorty","Zubair Shafiq"],"abstract":"There are concerns that the ability of language models (LMs) to generate high quality synthetic text can be misused to launch spam, disinformation, or propaganda. Therefore, the research community is actively working on developing approaches to detect whether a given text is organic or synthetic. While this is a useful first step, it is important to be able to further fingerprint the author LM to attribute its origin. Prior work on fingerprinting LMs is limited to attributing synthetic text generated by a handful (usually < 10) of pre-trained LMs. However, LMs such as GPT2 are commonly fine-tuned in a myriad of ways (e.g., on a domain-specific text corpus) before being used to generate synthetic text. It is challenging to fingerprinting fine-tuned LMs because the universe of fine-tuned LMs is much larger in realistic scenarios. To address this challenge, we study the problem of large-scale fingerprinting of fine-tuned LMs in the wild. Using a real-world dataset of synthetic text generated by 108 different fine-tuned LMs, we conduct comprehensive experiments to demonstrate the limitations of existing fingerprinting approaches. Our results show that fine-tuning itself is the most effective in attributing the synthetic text generated by fine-tuned LMs.","url_abs":"https://arxiv.org/abs/2106.01703v1","url_pdf":"https://arxiv.org/pdf/2106.01703v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fingerprinting-fine-tuned-language-models-in","repo_url":"https://github.com/LCS2-IIITD/ACL-FFLM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.01703","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.01703"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/LCS2-IIITD/ACL-FFLM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":16},"by_repo_kind":{"official":{"samples":16,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a34a68ec1f87d1c1","entry":"convert","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/generate-embed/pretrained/pretrained-generate-embed.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/generate-embed/pretrained/pretrained-generate-embed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a34a68ec1f87d1c1"}},{"code_sha256_prefix":"2e298fac74427689","entry":"convert_timestamp","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"preprocess/create_full_dataset.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/preprocess/create_full_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e298fac74427689"}},{"code_sha256_prefix":"a3aad5d3c0c10547","entry":"generate_embeddings","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/generate-embed/pretrained/pretrained-generate-embed.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/generate-embed/pretrained/pretrained-generate-embed.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3aad5d3c0c10547"}},{"code_sha256_prefix":"0d7d5e73ca198d40","entry":"get_dist","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/generate-embed/ft/seq_classification.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/generate-embed/ft/seq_classification.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0d7d5e73ca198d40"}},{"code_sha256_prefix":"217dfb86b70c79c5","entry":"initialize_classifiers","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/sklearn_classifiers.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/sklearn_classifiers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"217dfb86b70c79c5"}},{"code_sha256_prefix":"65d81b283d6d14fe","entry":"loadJsonFile","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/read_write_functions.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/read_write_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65d81b283d6d14fe"}},{"code_sha256_prefix":"d069cf3bd43db648","entry":"loadPickleFile","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/read_write_functions.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/read_write_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d069cf3bd43db648"}},{"code_sha256_prefix":"f940e406bb65ae47","entry":"loadPickleFile","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"create-splits/create_splits.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/create-splits/create_splits.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f940e406bb65ae47"}},{"code_sha256_prefix":"ddffc0e8663801cd","entry":"load_dataset","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/generate-embed/ft/seq_classification.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/generate-embed/ft/seq_classification.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ddffc0e8663801cd"}},{"code_sha256_prefix":"728e40674f6bf35c","entry":"make_dataset","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/generate-embed/ft/seq_classification.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/generate-embed/ft/seq_classification.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"728e40674f6bf35c"}},{"code_sha256_prefix":"0cc7fd039d01bf50","entry":"make_paths","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/classifiers/pretrained_classify_main_final.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/classifiers/pretrained_classify_main_final.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0cc7fd039d01bf50"}},{"code_sha256_prefix":"14a777d170409d8d","entry":"make_sets","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"create-splits/create_splits.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/create-splits/create_splits.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14a777d170409d8d"}},{"code_sha256_prefix":"246180ab9a03e813","entry":"new_classifier","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/sklearn_classifiers.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/sklearn_classifiers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"246180ab9a03e813"}},{"code_sha256_prefix":"656cc72eb9b46d98","entry":"openCSVfile","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"models/read_write_functions.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/models/read_write_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"656cc72eb9b46d98"}},{"code_sha256_prefix":"51db9a13268db992","entry":"openCSVfile","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"preprocess/create_full_dataset.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/preprocess/create_full_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51db9a13268db992"}},{"code_sha256_prefix":"61dfa1c9e178c8b1","entry":"tokenize_words","repo":"LCS2-IIITD/ACL-FFLM","repo_kind":"official","path":"preprocess/create_full_dataset.py","file_url":"https://github.com/LCS2-IIITD/ACL-FFLM/blob/HEAD/preprocess/create_full_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"61dfa1c9e178c8b1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}