{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/watching-the-ai-watchdogs-a-fairness-and","title":"Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers","arxiv_id":"2501.13302","date":"2025-01-23","proceeding":null,"authors":["Akshit Achara","Anshuman Chhabra"],"abstract":"AI Safety Moderation (ASM) classifiers are designed to moderate content on social media platforms and to serve as guardrails that prevent Large Language Models (LLMs) from being fine-tuned on unsafe inputs. Owing to their potential for disparate impact, it is crucial to ensure that these classifiers: (1) do not unfairly classify content belonging to users from minority groups as unsafe compared to those from majority groups and (2) that their behavior remains robust and consistent across similar inputs. In this work, we thus examine the fairness and robustness of four widely-used, closed-source ASM classifiers: OpenAI Moderation API, Perspective API, Google Cloud Natural Language (GCNL) API, and Clarifai API. We assess fairness using metrics such as demographic parity and conditional statistical parity, comparing their performance against ASM models and a fair-only baseline. Additionally, we analyze robustness by testing the classifiers' sensitivity to small and natural input perturbations. Our findings reveal potential fairness and robustness gaps, highlighting the need to mitigate these issues in future versions of these models.","url_abs":"https://arxiv.org/abs/2501.13302v1","url_pdf":"https://arxiv.org/pdf/2501.13302v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"watching-the-ai-watchdogs-a-fairness-and","repo_url":"https://github.com/acharaakshit/fairmod","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2501.13302","atlas_url":"https://app.syntology.ai/?focus=2501.13302","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.13302"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/acharaakshit/fairmod","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/acharaakshit/FairMod","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":5},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ec7977b6ec96dce1","entry":"chunker","repo":"acharaakshit/FairMod","repo_kind":"official","path":"utils.py","file_url":"https://github.com/acharaakshit/FairMod/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec7977b6ec96dce1"}},{"code_sha256_prefix":"86c8842f1015e8f1","entry":"conditional_statistical_parity","repo":"acharaakshit/FairMod","repo_kind":"official","path":"fairness_metrics.py","file_url":"https://github.com/acharaakshit/FairMod/blob/HEAD/fairness_metrics.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"86c8842f1015e8f1"}},{"code_sha256_prefix":"6896889548edd444","entry":"demographic_parity","repo":"acharaakshit/FairMod","repo_kind":"official","path":"fairness_metrics.py","file_url":"https://github.com/acharaakshit/FairMod/blob/HEAD/fairness_metrics.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6896889548edd444"}},{"code_sha256_prefix":"dba874d54d23eadf","entry":"modAPI","repo":"acharaakshit/FairMod","repo_kind":"official","path":"utils.py","file_url":"https://github.com/acharaakshit/FairMod/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dba874d54d23eadf"}},{"code_sha256_prefix":"832c2aa62967a652","entry":"process_dataset","repo":"acharaakshit/FairMod","repo_kind":"official","path":"utils.py","file_url":"https://github.com/acharaakshit/FairMod/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"832c2aa62967a652"}},{"code_sha256_prefix":"09a5cad73788379c","entry":"robustness","repo":"acharaakshit/FairMod","repo_kind":"official","path":"fairness_metrics.py","file_url":"https://github.com/acharaakshit/FairMod/blob/HEAD/fairness_metrics.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09a5cad73788379c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}