{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cover-learning-covariate-specific-vector","title":"CoVeR: Learning Covariate-Specific Vector Representations with Tensor Decompositions","arxiv_id":"1802.07839","date":"2018-02-21","proceeding":"ICML 2018 7","authors":["Kevin Tian","Teng Zhang","James Zou"],"abstract":"Word embedding is a useful approach to capture co-occurrence structures in\nlarge text corpora. However, in addition to the text data itself, we often have\nadditional covariates associated with individual corpus documents---e.g. the\ndemographic of the author, time and venue of publication---and we would like\nthe embedding to naturally capture this information. We propose CoVeR, a new\ntensor decomposition model for vector embeddings with covariates. CoVeR jointly\nlearns a \\emph{base} embedding for all the words as well as a weighted diagonal\nmatrix to model how each covariate affects the base embedding. To obtain author\nor venue-specific embedding, for example, we can then simply multiply the base\nembedding by the associated transformation matrix. The main advantages of our\napproach are data efficiency and interpretability of the covariate\ntransformation. Our experiments demonstrate that our joint model learns\nsubstantially better covariate-specific embeddings compared to the standard\napproach of learning a separate embedding for each covariate using only the\nrelevant subset of data, as well as other related methods. Furthermore, CoVeR\nencourages the embeddings to be \"topic-aligned\" in that the dimensions have\nspecific independent meanings. This allows our covariate-specific embeddings to\nbe compared by topic, enabling downstream differential analysis. We empirically\nevaluate the benefits of our algorithm on datasets, and demonstrate how it can\nbe used to address many natural questions about covariate effects.\n  Accompanying code to this paper can be found at\nhttp://github.com/kjtian/CoVeR.","url_abs":"http://arxiv.org/abs/1802.07839v2","url_pdf":"http://arxiv.org/pdf/1802.07839v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cover-learning-covariate-specific-vector","repo_url":"https://github.com/justinaL/tag","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"natural-questions","task_name":"Natural Questions"},{"task_slug":"tensor-decomposition","task_name":"Tensor Decomposition"}],"methods":[{"method_slug":"interpretability","method_name":"Interpretability"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}