{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/seqmia-sequential-metric-based-membership","title":"SeqMIA: Sequential-Metric Based Membership Inference Attack","arxiv_id":"2407.15098","date":"2024-07-21","proceeding":null,"authors":["Hao Li","Zheng Li","Siyuan Wu","Chengrui Hu","Yutong Ye","Min Zhang","Dengguo Feng","Yang Zhang"],"abstract":"Most existing membership inference attacks (MIAs) utilize metrics (e.g., loss) calculated on the model's final state, while recent advanced attacks leverage metrics computed at various stages, including both intermediate and final stages, throughout the model training. Nevertheless, these attacks often process multiple intermediate states of the metric independently, ignoring their time-dependent patterns. Consequently, they struggle to effectively distinguish between members and non-members who exhibit similar metric values, particularly resulting in a high false-positive rate. In this study, we delve deeper into the new membership signals in the black-box scenario. We identify a new, more integrated membership signal: the Pattern of Metric Sequence, derived from the various stages of model training. We contend that current signals provide only partial perspectives of this new signal: the new one encompasses both the model's multiple intermediate and final states, with a greater emphasis on temporal patterns among them. Building upon this signal, we introduce a novel attack method called Sequential-metric based Membership Inference Attack (SeqMIA). Specifically, we utilize knowledge distillation to obtain a set of distilled models representing various stages of the target model's training. We then assess multiple metrics on these distilled models in chronological order, creating distilled metric sequence. We finally integrate distilled multi-metric sequences as a sequential multiformat and employ an attention-based RNN attack model for inference. Empirical results show SeqMIA outperforms all baselines, especially can achieve an order of magnitude improvement in terms of TPR @ 0.1% FPR. Furthermore, we delve into the reasons why this signal contributes to SeqMIA's high attack performance, and assess various defense mechanisms against SeqMIA.","url_abs":"https://arxiv.org/abs/2407.15098v1","url_pdf":"https://arxiv.org/pdf/2407.15098v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"seqmia-sequential-metric-based-membership","repo_url":"https://github.com/AIPAG/SeqMIA","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"inference-attack","task_name":"Inference Attack"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"membership-inference-attack","task_name":"Membership Inference Attack"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2407.15098","atlas_url":"https://app.syntology.ai/?focus=2407.15098","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.15098"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AIPAG/SeqMIA","reach":{"status":"ok"}}],"summary":{"ran":9,"unverified":1},"by_repo_kind":{"official":{"samples":10,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"7409e0c223af763b","entry":"collate_fn","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/Models.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/Models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7409e0c223af763b"}},{"code_sha256_prefix":"375e951b2af83d40","entry":"createLossTrajectories_Seq","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/MetricSequence.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/MetricSequence.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"375e951b2af83d40"}},{"code_sha256_prefix":"52aaf83457a9896e","entry":"createMetricSequences","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/MetricSequence.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/MetricSequence.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52aaf83457a9896e"}},{"code_sha256_prefix":"dbbe96b6a193de26","entry":"getIndexByValue","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/attackMethodsFramework.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/attackMethodsFramework.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dbbe96b6a193de26"}},{"code_sha256_prefix":"777e34c7d997d0e5","entry":"get_file_name","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/SeqMIA.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/SeqMIA.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"777e34c7d997d0e5"}},{"code_sha256_prefix":"7fd673e4bf2cfcb2","entry":"load_data_for_trainortest","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/attackMethodsFramework.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/attackMethodsFramework.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7fd673e4bf2cfcb2"}},{"code_sha256_prefix":"1c76ac1e0eb2df4a","entry":"readCIFAR10","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/readData.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/readData.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c76ac1e0eb2df4a"}},{"code_sha256_prefix":"ab69149a1f441513","entry":"rescale","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/readData.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/readData.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab69149a1f441513"}},{"code_sha256_prefix":"3f36ea65add5e05b","entry":"reshape_for_save","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/readData.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/readData.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f36ea65add5e05b"}},{"code_sha256_prefix":"fcac9a0370e9fa95","entry":"load_data_for_attack","repo":"AIPAG/SeqMIA","repo_kind":"official","path":"SeqMIA/attackMethodsFramework.py","file_url":"https://github.com/AIPAG/SeqMIA/blob/HEAD/SeqMIA/attackMethodsFramework.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fcac9a0370e9fa95"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}