{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-price-of-strategyproofing-peer-assessment","title":"Strategyproofing Peer Assessment via Partitioning: The Price in Terms of Evaluators' Expertise","arxiv_id":"2201.10631","date":"2022-01-25","proceeding":null,"authors":["Komal Dhull","Steven Jecmen","Pravesh Kothari","Nihar B. Shah"],"abstract":"Strategic behavior is a fundamental problem in a variety of real-world applications that require some form of peer assessment, such as peer grading of homeworks, grant proposal review, conference peer review of scientific papers, and peer assessment of employees in organizations. Since an individual's own work is in competition with the submissions they are evaluating, they may provide dishonest evaluations to increase the relative standing of their own submission. This issue is typically addressed by partitioning the individuals and assigning them to evaluate the work of only those from different subsets. Although this method ensures strategyproofness, each submission may require a different type of expertise for effective evaluation. In this paper, we focus on finding an assignment of evaluators to submissions that maximizes assigned evaluators' expertise subject to the constraint of strategyproofness. We analyze the price of strategyproofness: that is, the amount of compromise on the assigned evaluators' expertise required in order to get strategyproofness. We establish several polynomial-time algorithms for strategyproof assignment along with assignment-quality guarantees. Finally, we evaluate the methods on a dataset from conference peer review.","url_abs":"https://arxiv.org/abs/2201.10631v3","url_pdf":"https://arxiv.org/pdf/2201.10631v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-price-of-strategyproofing-peer-assessment","repo_url":"https://github.com/sjecmen/optimal_strategyproof_assignment","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2201.10631","atlas_url":"https://app.syntology.ai/?focus=2201.10631","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.10631"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sjecmen/optimal_strategyproof_assignment","reach":null}],"summary":{"ran_draft_wrong":3,"ran_honours":2},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"cb7f867cfae25c9b","entry":"get_connected_components","repo":"sjecmen/optimal_strategyproof_assignment","repo_kind":"official","path":"main_general_authorship.py","file_url":"https://github.com/sjecmen/optimal_strategyproof_assignment/blob/HEAD/main_general_authorship.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb7f867cfae25c9b"}},{"code_sha256_prefix":"f75c0ce5f2217624","entry":"get_outcome_counts","repo":"sjecmen/optimal_strategyproof_assignment","repo_kind":"official","path":"evaluate_results.py","file_url":"https://github.com/sjecmen/optimal_strategyproof_assignment/blob/HEAD/evaluate_results.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f75c0ce5f2217624"}},{"code_sha256_prefix":"51431eae45a4a1f8","entry":"get_outcomes","repo":"sjecmen/optimal_strategyproof_assignment","repo_kind":"official","path":"evaluate_results.py","file_url":"https://github.com/sjecmen/optimal_strategyproof_assignment/blob/HEAD/evaluate_results.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"51431eae45a4a1f8"}},{"code_sha256_prefix":"45e19c421aa61cea","entry":"get_score_split","repo":"sjecmen/optimal_strategyproof_assignment","repo_kind":"official","path":"evaluate_results.py","file_url":"https://github.com/sjecmen/optimal_strategyproof_assignment/blob/HEAD/evaluate_results.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45e19c421aa61cea"}},{"code_sha256_prefix":"601b91a120941808","entry":"random_general_partition","repo":"sjecmen/optimal_strategyproof_assignment","repo_kind":"official","path":"main_general_authorship.py","file_url":"https://github.com/sjecmen/optimal_strategyproof_assignment/blob/HEAD/main_general_authorship.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"601b91a120941808"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}