{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/probabilistic-multigraph-modeling-for","title":"Probabilistic Multigraph Modeling for Improving the Quality of Crowdsourced Affective Data","arxiv_id":"1701.01096","date":"2017-01-04","proceeding":null,"authors":["Jianbo Ye","Jia Li","Michelle G. Newman","Reginald B. Adams, Jr.","James Z. Wang"],"abstract":"We proposed a probabilistic approach to joint modeling of participants'\nreliability and humans' regularity in crowdsourced affective studies.\nReliability measures how likely a subject will respond to a question seriously;\nand regularity measures how often a human will agree with other\nseriously-entered responses coming from a targeted population.\nCrowdsourcing-based studies or experiments, which rely on human self-reported\naffect, pose additional challenges as compared with typical crowdsourcing\nstudies that attempt to acquire concrete non-affective labels of objects. The\nreliability of participants has been massively pursued for typical\nnon-affective crowdsourcing studies, whereas the regularity of humans in an\naffective experiment in its own right has not been thoroughly considered. It\nhas been often observed that different individuals exhibit different feelings\non the same test question, which does not have a sole correct response in the\nfirst place. High reliability of responses from one individual thus cannot\nconclusively result in high consensus across individuals. Instead, globally\ntesting consensus of a population is of interest to investigators. Built upon\nthe agreement multigraph among tasks and workers, our probabilistic model\ndifferentiates subject regularity from population reliability. We demonstrate\nthe method's effectiveness for in-depth robust analysis of large-scale\ncrowdsourced affective data, including emotion and aesthetic assessments\ncollected by presenting visual stimuli to human subjects.","url_abs":"http://arxiv.org/abs/1701.01096v2","url_pdf":"http://arxiv.org/pdf/1701.01096v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"probabilistic-multigraph-modeling-for","repo_url":"https://github.com/bobye/GLBA","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1701.01096","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}