{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/antipodes-of-label-differential-privacy-pate","title":"Antipodes of Label Differential Privacy: PATE and ALIBI","arxiv_id":"2106.03408","date":"2021-06-07","proceeding":"NeurIPS 2021 12","authors":["Mani Malek","Ilya Mironov","Karthik Prasad","Igor Shilov","Florian Tramèr"],"abstract":"We consider the privacy-preserving machine learning (ML) setting where the trained model must satisfy differential privacy (DP) with respect to the labels of the training examples. We propose two novel approaches based on, respectively, the Laplace mechanism and the PATE framework, and demonstrate their effectiveness on standard benchmarks. While recent work by Ghazi et al. proposed Label DP schemes based on a randomized response mechanism, we argue that additive Laplace noise coupled with Bayesian inference (ALIBI) is a better fit for typical ML tasks. Moreover, we show how to achieve very strong privacy levels in some regimes, with our adaptation of the PATE framework that builds on recent advances in semi-supervised learning. We complement theoretical analysis of our algorithms' privacy guarantees with empirical evaluation of their memorization properties. Our evaluation suggests that comparing different algorithms according to their provable DP guarantees can be misleading and favor a less private algorithm with a tighter analysis. Code for implementation of algorithms and memorization attacks is available from https://github.com/facebookresearch/label_dp_antipodes.","url_abs":"https://arxiv.org/abs/2106.03408v2","url_pdf":"https://arxiv.org/pdf/2106.03408v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"antipodes-of-label-differential-privacy-pate","repo_url":"https://github.com/facebookresearch/label_dp_antipodes","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"memorization","task_name":"Memorization"},{"task_slug":"privacy-preserving","task_name":"Privacy Preserving"},{"task_slug":"privacy-preserving-deep-learning","task_name":"Privacy Preserving Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2106.03408","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.03408"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/facebookresearch/label_dp_antipodes","reach":null}],"summary":{"ran_honours":5,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1},"community":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"409c700e1f84cf5e","entry":"eps","repo":"facebookresearch/label_dp_antipodes","repo_kind":"official","path":"memorization_attack/attack.py","file_url":"https://github.com/facebookresearch/label_dp_antipodes/blob/HEAD/memorization_attack/attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"409c700e1f84cf5e"}},{"code_sha256_prefix":"764c1d4ec8bb2454","entry":"build_dynamically_alibi_tensor","repo":"seanzhang-zhichen/baichuan-Dynamic-NTK-ALiBi","repo_kind":"community","path":"modeling_baichuan.py","file_url":"https://github.com/seanzhang-zhichen/baichuan-Dynamic-NTK-ALiBi/blob/HEAD/modeling_baichuan.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"764c1d4ec8bb2454"}},{"code_sha256_prefix":"197348cafc1e4550","entry":"build_ntk_alibi_tensor","repo":"seanzhang-zhichen/baichuan-Dynamic-NTK-ALiBi","repo_kind":"community","path":"modeling_baichuan.py","file_url":"https://github.com/seanzhang-zhichen/baichuan-Dynamic-NTK-ALiBi/blob/HEAD/modeling_baichuan.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"197348cafc1e4550"}},{"code_sha256_prefix":"4dbdfa03dac7f311","entry":"compute_eps","repo":"facebookresearch/label_dp_antipodes","repo_kind":"official","path":"memorization_attack/attack.py","file_url":"https://github.com/facebookresearch/label_dp_antipodes/blob/HEAD/memorization_attack/attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4dbdfa03dac7f311"}},{"code_sha256_prefix":"f1d8454322a9be95","entry":"rand_pos_and_labels","repo":"facebookresearch/label_dp_antipodes","repo_kind":"official","path":"memorization_attack/attack.py","file_url":"https://github.com/facebookresearch/label_dp_antipodes/blob/HEAD/memorization_attack/attack.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f1d8454322a9be95"}},{"code_sha256_prefix":"fba6228bff78a52b","entry":"train","repo":"facebookresearch/label_dp_antipodes","repo_kind":"official","path":"train_cifar_alibi.py","file_url":"https://github.com/facebookresearch/label_dp_antipodes/blob/HEAD/train_cifar_alibi.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fba6228bff78a52b"}},{"code_sha256_prefix":"42fa1baaddd7b892","entry":"accuracy","repo":"facebookresearch/label_dp_antipodes","repo_kind":"official","path":"train_cifar_alibi.py","file_url":"https://github.com/facebookresearch/label_dp_antipodes/blob/HEAD/train_cifar_alibi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"42fa1baaddd7b892"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}