{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/comparative-evaluation-of-label-agnostic","title":"Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets","arxiv_id":null,"date":"2020-11-01","proceeding":"EMNLP 2020 11","authors":["Nedjma Ousidhoum","Yangqiu Song","Dit-yan Yeung"],"abstract":"Work on bias in hate speech typically aims to improve classification performance while relatively overlooking the quality of the data. We examine selection bias in hate speech in a language and label independent fashion. We first use topic models to discover latent semantics in eleven hate speech corpora, then, we present two bias evaluation metrics based on the semantic similarity between topics and search words frequently used to build corpora. We discuss the possibility of revising the data collection process by comparing datasets and analyzing contrastive case studies.","url_abs":"https://aclanthology.org/2020.emnlp-main.199","url_pdf":"https://aclanthology.org/2020.emnlp-main.199.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"comparative-evaluation-of-label-agnostic","repo_url":"https://github.com/HKUST-KnowComp/HS_Bias_Eval","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"hate-speech-detection","task_name":"Hate Speech Detection"},{"task_slug":"selection-bias","task_name":"Selection bias"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"},{"task_slug":"topic-models","task_name":"Topic Models"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}