{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/how-useful-are-reviews-for-recommendation-a","title":"How Useful are Reviews for Recommendation? A Critical Review and Potential Improvements","arxiv_id":"2005.12210","date":"2020-05-25","proceeding":null,"authors":["Noveen Sachdeva","Julian McAuley"],"abstract":"We investigate a growing body of work that seeks to improve recommender systems through the use of review text. Generally, these papers argue that since reviews 'explain' users' opinions, they ought to be useful to infer the underlying dimensions that predict ratings or purchases. Schemes to incorporate reviews range from simple regularizers to neural network approaches. Our initial findings reveal several discrepancies in reported results, partly due to (e.g.) copying results across papers despite changes in experimental settings or data pre-processing. First, we attempt a comprehensive analysis to resolve these ambiguities. Further investigation calls for discussion on a much larger problem about the \"importance\" of user reviews for recommendation. Through a wide range of experiments, we observe several cases where state-of-the-art methods fail to outperform existing baselines, especially as we deviate from a few narrowly-defined settings where reviews are useful. We conclude by providing hypotheses for our observations, that seek to characterize under what conditions reviews are likely to be helpful. Through this work, we aim to evaluate the direction in which the field is progressing and encourage robust empirical evaluation.","url_abs":"https://arxiv.org/abs/2005.12210v1","url_pdf":"https://arxiv.org/pdf/2005.12210v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"how-useful-are-reviews-for-recommendation-a","repo_url":"https://github.com/noveens/reviews4rec","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2005.12210","atlas_url":"https://app.syntology.ai/?focus=2005.12210","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2005.12210"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/noveens/reviews4rec","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b91a5961fb87cb5e","entry":"load_obj","repo":"noveens/reviews4rec","repo_kind":"official","path":"utils.py","file_url":"https://github.com/noveens/reviews4rec/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b91a5961fb87cb5e"}},{"code_sha256_prefix":"788485aabd49f5c7","entry":"load_data_fast","repo":"noveens/reviews4rec","repo_kind":"official","path":"data_fast.py","file_url":"https://github.com/noveens/reviews4rec/blob/HEAD/data_fast.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"788485aabd49f5c7"}},{"code_sha256_prefix":"eb7cbc68b98e37a0","entry":"load_obj_json","repo":"noveens/reviews4rec","repo_kind":"official","path":"utils.py","file_url":"https://github.com/noveens/reviews4rec/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb7cbc68b98e37a0"}},{"code_sha256_prefix":"c3ce7ab247898b78","entry":"load_user_item_counts","repo":"noveens/reviews4rec","repo_kind":"official","path":"utils.py","file_url":"https://github.com/noveens/reviews4rec/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3ce7ab247898b78"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}