{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/qa-domain-adaptation-using-hidden-space","title":"QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation","arxiv_id":"2210.10861","date":"2022-10-19","proceeding":null,"authors":["Zhenrui Yue","Huimin Zeng","Bernhard Kratzwald","Stefan Feuerriegel","Dong Wang"],"abstract":"Question answering (QA) has recently shown impressive results for answering questions from customized domains. Yet, a common challenge is to adapt QA models to an unseen target domain. In this paper, we propose a novel self-supervised framework called QADA for QA domain adaptation. QADA introduces a novel data augmentation pipeline used to augment training QA samples. Different from existing methods, we enrich the samples via hidden space augmentation. For questions, we introduce multi-hop synonyms and sample augmented token embeddings with Dirichlet distributions. For contexts, we develop an augmentation method which learns to drop context spans via a custom attentive sampling strategy. Additionally, contrastive learning is integrated in the proposed self-supervised adaptation framework QADA. Unlike existing approaches, we generate pseudo labels and propose to train the model via a novel attention-based contrastive adaptation method. The attention weights are used to build informative features for discrepancy estimation that helps the QA model separate answers and generalize across source and target domains. To the best of our knowledge, our work is the first to leverage hidden space augmentation and attention-based contrastive adaptation for self-supervised domain adaptation in QA. Our evaluation shows that QADA achieves considerable improvements on multiple target datasets over state-of-the-art baselines in QA domain adaptation.","url_abs":"https://arxiv.org/abs/2210.10861v1","url_pdf":"https://arxiv.org/pdf/2210.10861v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"qa-domain-adaptation-using-hidden-space","repo_url":"https://github.com/yueeeeeeee/self-supervised-qa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.10861","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.10861"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Yueeeeeeee/Self-Supervised-QA","reach":null}],"summary":{"ran":2,"ran_honours":5,"ran_violates":1,"ran_draft_wrong":3,"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":11,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"39391d9b3635435c","entry":"BERTRandomSampler","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"39391d9b3635435c"}},{"code_sha256_prefix":"6fb95e09ee551a2f","entry":"BertAdam","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6fb95e09ee551a2f"}},{"code_sha256_prefix":"cd812ef970436cb7","entry":"compare_performance","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cd812ef970436cb7"}},{"code_sha256_prefix":"6e78b0bb4297c189","entry":"evaluate_acc_and_f1","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e78b0bb4297c189"}},{"code_sha256_prefix":"f6c275d6a18330a9","entry":"exact_match_score","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f6c275d6a18330a9"}},{"code_sha256_prefix":"2112c433b9c6d343","entry":"f1_score","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"2112c433b9c6d343"}},{"code_sha256_prefix":"e5f44823bc5e7f55","entry":"get_bert_model_parameters","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e5f44823bc5e7f55"}},{"code_sha256_prefix":"71946db113d7b29a","entry":"keep_high_prob_samples","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"71946db113d7b29a"}},{"code_sha256_prefix":"6829ceb72569fa12","entry":"metric_max_over_ground_truths","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"6829ceb72569fa12"}},{"code_sha256_prefix":"c6a80c065d2e4851","entry":"normalize_answer","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"c6a80c065d2e4851"}},{"code_sha256_prefix":"e7d542062316094a","entry":"warmup_constant","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e7d542062316094a"}},{"code_sha256_prefix":"c58d57224530d17e","entry":"warmup_linear","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"c58d57224530d17e"}},{"code_sha256_prefix":"e57c4b1183dcd6cb","entry":"comb_adversarial_training_stage","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e57c4b1183dcd6cb"}},{"code_sha256_prefix":"b0b6c98806c9fd6d","entry":"evaluation_stage","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0b6c98806c9fd6d"}},{"code_sha256_prefix":"4bdc939ce31c7cf4","entry":"generate_self_training_samples","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4bdc939ce31c7cf4"}},{"code_sha256_prefix":"35f7cddf90dd05d4","entry":"warmup_cosine","repo":"Yueeeeeeee/Self-Supervised-QA","repo_kind":"official","path":"src/run_qada.py","file_url":"https://github.com/Yueeeeeeee/Self-Supervised-QA/blob/HEAD/src/run_qada.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"35f7cddf90dd05d4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}