{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/subjqa-a-dataset-for-subjectivity-and-review","title":"SubjQA: A Dataset for Subjectivity and Review Comprehension","arxiv_id":"2004.14283","date":"2020-04-29","proceeding":"EMNLP 2020 11","authors":["Johannes Bjerva","Nikita Bhutani","Behzad Golshan","Wang-Chiew Tan","Isabelle Augenstein"],"abstract":"Subjectivity is the expression of internal opinions or beliefs which cannot be objectively observed or verified, and has been shown to be important for sentiment analysis and word-sense disambiguation. Furthermore, subjectivity is an important aspect of user-generated data. In spite of this, subjectivity has not been investigated in contexts where such data is widespread, such as in question answering (QA). We therefore investigate the relationship between subjectivity and QA, while developing a new dataset. We compare and contrast with analyses from previous work, and verify that findings regarding subjectivity still hold when using recently developed NLP architectures. We find that subjectivity is also an important feature in the case of QA, albeit with more intricate interactions between subjectivity and QA performance. For instance, a subjective question may or may not be associated with a subjective answer. We release an English QA dataset (SubjQA) based on customer reviews, containing subjectivity annotations for questions and answer spans across 6 distinct domains.","url_abs":"https://arxiv.org/abs/2004.14283v3","url_pdf":"https://arxiv.org/pdf/2004.14283v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"subjqa-a-dataset-for-subjectivity-and-review","repo_url":"https://github.com/megagonlabs/SubjQA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"word-sense-disambiguation","task_name":"Word Sense Disambiguation"}],"methods":[],"datasets_introduced":[{"slug":"subjqa","name":"SubjQA","full_name":null}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2004.14283","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2004.14283"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/uclnlp/jack","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":2},"by_repo_kind":{"found_in_text":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"827e812c314c4426","entry":"fetch_parents","repo":"uclnlp/jack","repo_kind":"found_in_text","path":"bin/jack-train.py","file_url":"https://github.com/uclnlp/jack/blob/HEAD/bin/jack-train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"827e812c314c4426"}},{"code_sha256_prefix":"4961fc4669cf2576","entry":"jack_to_qasetting","repo":"uclnlp/jack","repo_kind":"found_in_text","path":"jack/core/data_structures.py","file_url":"https://github.com/uclnlp/jack/blob/HEAD/jack/core/data_structures.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4961fc4669cf2576"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}