{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ethos-an-online-hate-speech-detection-dataset","title":"ETHOS: an Online Hate Speech Detection Dataset","arxiv_id":"2006.08328","date":"2020-06-11","proceeding":null,"authors":["Ioannis Mollas","Zoe Chrysopoulou","Stamatis Karlos","Grigorios Tsoumakas"],"abstract":"Online hate speech is a recent problem in our society that is rising at a steady pace by leveraging the vulnerabilities of the corresponding regimes that characterise most social media platforms. This phenomenon is primarily fostered by offensive comments, either during user interaction or in the form of a posted multimedia context. Nowadays, giant corporations own platforms where millions of users log in every day, and protection from exposure to similar phenomena appears to be necessary in order to comply with the corresponding legislation and maintain a high level of service quality. A robust and reliable system for detecting and preventing the uploading of relevant content will have a significant impact on our digitally interconnected society. Several aspects of our daily lives are undeniably linked to our social profiles, making us vulnerable to abusive behaviours. As a result, the lack of accurate hate speech detection mechanisms would severely degrade the overall user experience, although its erroneous operation would pose many ethical concerns. In this paper, we present 'ETHOS', a textual dataset with two variants: binary and multi-label, based on YouTube and Reddit comments validated using the Figure-Eight crowdsourcing platform. Furthermore, we present the annotation protocol used to create this dataset: an active sampling procedure for balancing our data in relation to the various aspects defined. Our key assumption is that, even gaining a small amount of labelled data from such a time-consuming process, we can guarantee hate speech occurrences in the examined material.","url_abs":"https://arxiv.org/abs/2006.08328v2","url_pdf":"https://arxiv.org/pdf/2006.08328v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ethos-an-online-hate-speech-detection-dataset","repo_url":"https://github.com/intelligence-csd-auth-gr/Ethos-Hate-Speech-Dataset","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"hate-speech-detection","task_name":"Hate Speech Detection"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"}],"datasets_introduced":[{"slug":"ethos","name":"ETHOS","full_name":"multi-labEl haTe speecH detectiOn dataSet"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/hate-speech-detection-on-ethos-binary","task":"Hate Speech Detection","dataset":"Ethos Binary","model":"BERT","rank_in_archive_order":2,"of":12,"metrics":{"Classification Accuracy":"0.7664","F1-score":"0.7883","Precision":"79.17"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-binary","task":"Hate Speech Detection","dataset":"Ethos Binary","model":"BiLSTM+Attention+FT","rank_in_archive_order":3,"of":12,"metrics":{"Classification Accuracy":"0.7734","F1-score":"0.768","Precision":"77.76"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-binary","task":"Hate Speech Detection","dataset":"Ethos Binary","model":"CNN+Attention+FT+GV","rank_in_archive_order":5,"of":12,"metrics":{"Classification Accuracy":"0.7515","F1-score":"0.7441","Precision":"74.92"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-binary","task":"Hate Speech Detection","dataset":"Ethos Binary","model":"SVM","rank_in_archive_order":8,"of":12,"metrics":{"Classification Accuracy":"0.6643","F1-score":"0.6607","Precision":"66.47"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-binary","task":"Hate Speech Detection","dataset":"Ethos Binary","model":"Random Forests","rank_in_archive_order":9,"of":12,"metrics":{"Classification Accuracy":"0.6504","F1-score":"0.6441","Precision":"64.69"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-multilabel","task":"Hate Speech Detection","dataset":"Ethos MultiLabel","model":"MLARAM","rank_in_archive_order":1,"of":5,"metrics":{"Hamming Loss":"0.2948"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-multilabel","task":"Hate Speech Detection","dataset":"Ethos MultiLabel","model":"MLkNN","rank_in_archive_order":2,"of":5,"metrics":{"Hamming Loss":"0.1606"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-multilabel","task":"Hate Speech Detection","dataset":"Ethos MultiLabel","model":"Binary Relevance","rank_in_archive_order":3,"of":5,"metrics":{"Hamming Loss":"0.1395"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-multilabel","task":"Hate Speech Detection","dataset":"Ethos MultiLabel","model":"Neural Classifier Chains","rank_in_archive_order":4,"of":5,"metrics":{"Hamming Loss":"0.132"},"uses_additional_data":false},{"leaderboard":"/sota/hate-speech-detection-on-ethos-multilabel","task":"Hate Speech Detection","dataset":"Ethos MultiLabel","model":"Neural Binary Relevance","rank_in_archive_order":5,"of":5,"metrics":{"Hamming Loss":"0.1097"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2006.08328","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}