{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/semi-supervised-knowledge-transfer-for-deep","title":"Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data","arxiv_id":"1610.05755","date":"2016-10-18","proceeding":null,"authors":["Nicolas Papernot","Martín Abadi","Úlfar Erlingsson","Ian Goodfellow","Kunal Talwar"],"abstract":"Some machine learning applications involve training data that is sensitive,\nsuch as the medical histories of patients in a clinical trial. A model may\ninadvertently and implicitly store some of its training data; careful analysis\nof the model may therefore reveal sensitive information.\n  To address this problem, we demonstrate a generally applicable approach to\nproviding strong privacy guarantees for training data: Private Aggregation of\nTeacher Ensembles (PATE). The approach combines, in a black-box fashion,\nmultiple models trained with disjoint datasets, such as records from different\nsubsets of users. Because they rely directly on sensitive data, these models\nare not published, but instead used as \"teachers\" for a \"student\" model. The\nstudent learns to predict an output chosen by noisy voting among all of the\nteachers, and cannot directly access an individual teacher or the underlying\ndata or parameters. The student's privacy properties can be understood both\nintuitively (since no single teacher and thus no single dataset dictates the\nstudent's training) and formally, in terms of differential privacy. These\nproperties hold even if an adversary can not only query the student but also\ninspect its internal workings.\n  Compared with previous work, the approach imposes only weak assumptions on\nhow teachers are trained: it applies to any model, including non-convex models\nlike DNNs. We achieve state-of-the-art privacy/utility trade-offs on MNIST and\nSVHN thanks to an improved privacy analysis and semi-supervised learning.","url_abs":"http://arxiv.org/abs/1610.05755v4","url_pdf":"http://arxiv.org/pdf/1610.05755v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"semi-supervised-knowledge-transfer-for-deep","repo_url":"https://github.com/tensorflow/privacy/tree/master/research/pate_2017","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null},{"paper_slug":"semi-supervised-knowledge-transfer-for-deep","repo_url":"https://github.com/aristizabal95/Making-PATE-Bidirecitonally-Private","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"semi-supervised-knowledge-transfer-for-deep","repo_url":"https://github.com/aristizabal95/Making-PATE-Bidirectionally-Private","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"semi-supervised-knowledge-transfer-for-deep","repo_url":"https://github.com/ashishdandekar/Privacy-at-risk","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"semi-supervised-knowledge-transfer-for-deep","repo_url":"https://github.com/chun-wei-ho/private-speech-adapter","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"semi-supervised-knowledge-transfer-for-deep","repo_url":"https://github.com/dimun/pate_torch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"semi-supervised-knowledge-transfer-for-deep","repo_url":"https://github.com/hrsu/multiple_teachers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"semi-supervised-knowledge-transfer-for-deep","repo_url":"https://github.com/kamathhrishi/PATE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1610.05755","atlas_url":"https://app.syntology.ai/?focus=1610.05755","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1610.05755"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kamathhrishi/PATE","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ashishdandekar/Privacy-at-risk","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chun-wei-ho/private-speech-adapter","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dimun/pate_torch","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aristizabal95/Making-PATE-Bidirecitonally-Private","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aristizabal95/Making-PATE-Bidirectionally-Private","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/privacy/tree/master/research/pate_2017","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hrsu/multiple_teachers","reach":null}],"summary":{"ran_honours":3,"ran_fixture":5,"ran_draft_wrong":1},"by_repo_kind":{"listed":{"samples":9,"ran":9,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"7644dcc3b6f870e3","entry":"PAR","repo":"ashishdandekar/Privacy-at-risk","repo_kind":"listed","path":"PATE/analysis.py","file_url":"https://github.com/ashishdandekar/Privacy-at-risk/blob/HEAD/PATE/analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"7644dcc3b6f870e3"}},{"code_sha256_prefix":"018c271d25bb072f","entry":"build_logistic","repo":"ashishdandekar/Privacy-at-risk","repo_kind":"listed","path":"PATE/teachers.py","file_url":"https://github.com/ashishdandekar/Privacy-at-risk/blob/HEAD/PATE/teachers.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"018c271d25bb072f"}},{"code_sha256_prefix":"a3dad74681c7b99c","entry":"build_model","repo":"ashishdandekar/Privacy-at-risk","repo_kind":"listed","path":"PATE/student.py","file_url":"https://github.com/ashishdandekar/Privacy-at-risk/blob/HEAD/PATE/student.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a3dad74681c7b99c"}},{"code_sha256_prefix":"c398a6682731556b","entry":"compute_q_noisy_max","repo":"ashishdandekar/Privacy-at-risk","repo_kind":"listed","path":"PATE/analysis.py","file_url":"https://github.com/ashishdandekar/Privacy-at-risk/blob/HEAD/PATE/analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"c398a6682731556b"}},{"code_sha256_prefix":"581c73d2f9b79b83","entry":"disturb","repo":"hrsu/multiple_teachers","repo_kind":"listed","path":"differential_privacy1/differential_privacy/multiple_teachers/train_teachers.py","file_url":"https://github.com/hrsu/multiple_teachers/blob/HEAD/differential_privacy1/differential_privacy/multiple_teachers/train_teachers.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"581c73d2f9b79b83"}},{"code_sha256_prefix":"68449dee88891420","entry":"generate_dataset","repo":"ashishdandekar/Privacy-at-risk","repo_kind":"listed","path":"PATE/student.py","file_url":"https://github.com/ashishdandekar/Privacy-at-risk/blob/HEAD/PATE/student.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"68449dee88891420"}},{"code_sha256_prefix":"be23fce1736697e8","entry":"logmgf_exact","repo":"ashishdandekar/Privacy-at-risk","repo_kind":"listed","path":"PATE/analysis.py","file_url":"https://github.com/ashishdandekar/Privacy-at-risk/blob/HEAD/PATE/analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"be23fce1736697e8"}},{"code_sha256_prefix":"506614109988e897","entry":"shuffle_data","repo":"ashishdandekar/Privacy-at-risk","repo_kind":"listed","path":"PATE/teachers.py","file_url":"https://github.com/ashishdandekar/Privacy-at-risk/blob/HEAD/PATE/teachers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"506614109988e897"}},{"code_sha256_prefix":"b1ab84060a406b17","entry":"split_data","repo":"ashishdandekar/Privacy-at-risk","repo_kind":"listed","path":"PATE/teachers.py","file_url":"https://github.com/ashishdandekar/Privacy-at-risk/blob/HEAD/PATE/teachers.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b1ab84060a406b17"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}