{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/transferring-bert-like-transformers-knowledge","title":"Rethinking the Authorship Verification Experimental Setups","arxiv_id":"2112.05125","date":"2021-12-09","proceeding":null,"authors":["Florin Brad","Andrei Manolache","Elena Burceanu","Antonio Barbalau","Radu Ionescu","Marius Popescu"],"abstract":"One of the main drivers of the recent advances in authorship verification is the PAN large-scale authorship dataset. Despite generating significant progress in the field, inconsistent performance differences between the closed and open test sets have been reported. To this end, we improve the experimental setup by proposing five new public splits over the PAN dataset, specifically designed to isolate and identify biases related to the text topic and to the author's writing style. We evaluate several BERT-like baselines on these splits, showing that such models are competitive with authorship verification state-of-the-art methods. Furthermore, using explainable AI, we find that these baselines are biased towards named entities. We show that models trained without the named entities obtain better results and generalize better when tested on DarkReddit, our new dataset for authorship verification.","url_abs":"https://arxiv.org/abs/2112.05125v2","url_pdf":"https://arxiv.org/pdf/2112.05125v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"transferring-bert-like-transformers-knowledge","repo_url":"https://github.com/bit-ml/dupin","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"authorship-verification","task_name":"Authorship Verification"},{"task_slug":"domain-generalization","task_name":"Domain Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.05125","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.05125"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bit-ml/dupin","reach":null}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"01f03b73eac7b9e4","entry":"get_authors_data_from_folder","repo":"bit-ml/dupin","repo_kind":"official","path":"preprocess/split_train_val.py","file_url":"https://github.com/bit-ml/dupin/blob/HEAD/preprocess/split_train_val.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"01f03b73eac7b9e4"}},{"code_sha256_prefix":"9addd89f05b09a48","entry":"read_jsonl_examples","repo":"bit-ml/dupin","repo_kind":"official","path":"preprocess/split_train_val.py","file_url":"https://github.com/bit-ml/dupin/blob/HEAD/preprocess/split_train_val.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9addd89f05b09a48"}},{"code_sha256_prefix":"a018e25f6d3be825","entry":"split_pan_dataset_closed_set_v1","repo":"bit-ml/dupin","repo_kind":"official","path":"preprocess/split_train_val.py","file_url":"https://github.com/bit-ml/dupin/blob/HEAD/preprocess/split_train_val.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a018e25f6d3be825"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}