{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scruples-a-corpus-of-community-ethical","title":"Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes","arxiv_id":"2008.09094","date":"2020-08-20","proceeding":null,"authors":["Nicholas Lourie","Ronan Le Bras","Yejin Choi"],"abstract":"As AI systems become an increasing part of people's everyday lives, it becomes ever more important that they understand people's ethical norms. Motivated by descriptive ethics, a field of study that focuses on people's descriptive judgments rather than theoretical prescriptions on morality, we investigate a novel, data-driven approach to machine ethics. We introduce Scruples, the first large-scale dataset with 625,000 ethical judgments over 32,000 real-life anecdotes. Each anecdote recounts a complex ethical situation, often posing moral dilemmas, paired with a distribution of judgments contributed by the community members. Our dataset presents a major challenge to state-of-the-art neural language models, leaving significant room for improvement. However, when presented with simplified moral situations, the results are considerably more promising, suggesting that neural models can effectively learn simpler ethical building blocks. A key take-away of our empirical analysis is that norms are not always clean-cut; many situations are naturally divisive. We present a new method to estimate the best possible performance on such tasks with inherently diverse label distributions, and explore likelihood functions that separate intrinsic from model uncertainty.","url_abs":"https://arxiv.org/abs/2008.09094v2","url_pdf":"https://arxiv.org/pdf/2008.09094v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scruples-a-corpus-of-community-ethical","repo_url":"https://github.com/allenai/scruples","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"descriptive","task_name":"Descriptive"},{"task_slug":"ethics","task_name":"Ethics"}],"methods":[],"datasets_introduced":[{"slug":"scruples","name":"Scruples","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2008.09094","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.09094"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/allenai/scruples","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d49c0507a4ee86ee","entry":"configure_logging","repo":"allenai/scruples","repo_kind":"official","path":"src/scruples/utils.py","file_url":"https://github.com/allenai/scruples/blob/HEAD/src/scruples/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d49c0507a4ee86ee"}},{"code_sha256_prefix":"d6c73ac35e8781c2","entry":"count_words","repo":"allenai/scruples","repo_kind":"official","path":"src/scruples/utils.py","file_url":"https://github.com/allenai/scruples/blob/HEAD/src/scruples/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d6c73ac35e8781c2"}},{"code_sha256_prefix":"4a8e5b9c7d1dd04d","entry":"xentropy","repo":"allenai/scruples","repo_kind":"official","path":"src/scruples/utils.py","file_url":"https://github.com/allenai/scruples/blob/HEAD/src/scruples/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a8e5b9c7d1dd04d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}