{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-difficulty-of-defending-self","title":"On the Difficulty of Defending Self-Supervised Learning against Model Extraction","arxiv_id":"2205.07890","date":"2022-05-16","proceeding":null,"authors":["Adam Dziedzic","Nikita Dhawan","Muhammad Ahmad Kaleem","Jonas Guan","Nicolas Papernot"],"abstract":"Self-Supervised Learning (SSL) is an increasingly popular ML paradigm that trains models to transform complex inputs into representations without relying on explicit labels. These representations encode similarity structures that enable efficient learning of multiple downstream tasks. Recently, ML-as-a-Service providers have commenced offering trained SSL models over inference APIs, which transform user inputs into useful representations for a fee. However, the high cost involved to train these models and their exposure over APIs both make black-box extraction a realistic security threat. We thus explore model stealing attacks against SSL. Unlike traditional model extraction on classifiers that output labels, the victim models here output representations; these representations are of significantly higher dimensionality compared to the low-dimensional prediction scores output by classifiers. We construct several novel attacks and find that approaches that train directly on a victim's stolen representations are query efficient and enable high accuracy for downstream models. We then show that existing defenses against model extraction are inadequate and not easily retrofitted to the specificities of SSL.","url_abs":"https://arxiv.org/abs/2205.07890v3","url_pdf":"https://arxiv.org/pdf/2205.07890v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-difficulty-of-defending-self","repo_url":"https://github.com/cleverhans-lab/ssl-attacks-defenses","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"model-extraction","task_name":"Model extraction"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2205.07890","atlas_url":"https://app.syntology.ai/?focus=2205.07890","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.07890"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cleverhans-lab/ssl-attacks-defenses","reach":null}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"728d0ad41f3fb7c5","entry":"load_stolen","repo":"cleverhans-lab/ssl-attacks-defenses","repo_kind":"official","path":"linear_eval.py","file_url":"https://github.com/cleverhans-lab/ssl-attacks-defenses/blob/HEAD/linear_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"728d0ad41f3fb7c5"}},{"code_sha256_prefix":"a7343548e1145053","entry":"load_victim","repo":"cleverhans-lab/ssl-attacks-defenses","repo_kind":"official","path":"linear_eval.py","file_url":"https://github.com/cleverhans-lab/ssl-attacks-defenses/blob/HEAD/linear_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7343548e1145053"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}