{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/calibration-with-privacy-in-peer-review","title":"Calibration with Privacy in Peer Review","arxiv_id":"2201.11308","date":"2022-01-27","proceeding":null,"authors":["Wenxin Ding","Gautam Kamath","Weina Wang","Nihar B. Shah"],"abstract":"Reviewers in peer review are often miscalibrated: they may be strict, lenient, extreme, moderate, etc. A number of algorithms have previously been proposed to calibrate reviews. Such attempts of calibration can however leak sensitive information about which reviewer reviewed which paper. In this paper, we identify this problem of calibration with privacy, and provide a foundational building block to address it. Specifically, we present a theoretical study of this problem under a simplified-yet-challenging model involving two reviewers, two papers, and an MAP-computing adversary. Our main results establish the Pareto frontier of the tradeoff between privacy (preventing the adversary from inferring reviewer identity) and utility (accepting better papers), and design explicit computationally-efficient algorithms that we prove are Pareto optimal.","url_abs":"https://arxiv.org/abs/2201.11308v1","url_pdf":"https://arxiv.org/pdf/2201.11308v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"calibration-with-privacy-in-peer-review","repo_url":"https://github.com/wenxind/calibration-with-privacy-in-peer-review","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2201.11308","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.11308"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wenxind/calibration-with-privacy-in-peer-review","reach":null}],"summary":{"ran_honours":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"b16e443250945d35","entry":"find_assignment","repo":"wenxind/calibration-with-privacy-in-peer-review","repo_kind":"official","path":"simulation.py","file_url":"https://github.com/wenxind/calibration-with-privacy-in-peer-review/blob/HEAD/simulation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b16e443250945d35"}},{"code_sha256_prefix":"bed5dea30f67143d","entry":"generate_assignment","repo":"wenxind/calibration-with-privacy-in-peer-review","repo_kind":"official","path":"simulation.py","file_url":"https://github.com/wenxind/calibration-with-privacy-in-peer-review/blob/HEAD/simulation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bed5dea30f67143d"}},{"code_sha256_prefix":"84d5ef45526ac962","entry":"valid_assignment","repo":"wenxind/calibration-with-privacy-in-peer-review","repo_kind":"official","path":"simulation.py","file_url":"https://github.com/wenxind/calibration-with-privacy-in-peer-review/blob/HEAD/simulation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"84d5ef45526ac962"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}