{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/categorical-metadata-representation-for","title":"Categorical Metadata Representation for Customized Text Classification","arxiv_id":"1902.05196","date":"2019-02-14","proceeding":"TACL 2019 3","authors":["Jihyeok Kim","Reinald Kim Amplayo","Kyungjae Lee","Sua Sung","Minji Seo","Seung-won Hwang"],"abstract":"The performance of text classification has improved tremendously using\nintelligently engineered neural-based models, especially those injecting\ncategorical metadata as additional information, e.g., using user/product\ninformation for sentiment classification. These information have been used to\nmodify parts of the model (e.g., word embeddings, attention mechanisms) such\nthat results can be customized according to the metadata. We observe that\ncurrent representation methods for categorical metadata, which are devised for\nhuman consumption, are not as effective as claimed in popular classification\nmethods, outperformed even by simple concatenation of categorical features in\nthe final layer of the sentence encoder. We conjecture that categorical\nfeatures are harder to represent for machine use, as available context only\nindirectly describes the category, and even such context is often scarce (for\ntail category). To this end, we propose to use basis vectors to effectively\nincorporate categorical metadata on various parts of a neural-based model. This\nadditionally decreases the number of parameters dramatically, especially when\nthe number of categorical features is large. Extensive experiments on various\ndatasets with different properties are performed and show that through our\nmethod, we can represent categorical metadata more effectively to customize\nparts of the model, including unexplored ones, and increase the performance of\nthe model greatly.","url_abs":"http://arxiv.org/abs/1902.05196v1","url_pdf":"http://arxiv.org/pdf/1902.05196v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"categorical-metadata-representation-for","repo_url":"https://github.com/zizi1532/BasisCustomize","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"categorical-metadata-representation-for","repo_url":"https://github.com/zhou059/w266-project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"sentiment-classification","task_name":"Sentiment Classification"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sentiment-analysis-on-user-and-product","task":"Sentiment Analysis","dataset":"User and product information","model":"BiLSTM + linear-basis-cust","rank_in_archive_order":10,"of":10,"metrics":{"Yelp 2013 (Acc)":"67.1"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1902.05196","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}