{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/active-discriminative-text-representation","title":"Active Discriminative Text Representation Learning","arxiv_id":"1606.04212","date":"2016-06-14","proceeding":null,"authors":["Ye Zhang","Matthew Lease","Byron C. Wallace"],"abstract":"We propose a new active learning (AL) method for text classification with\nconvolutional neural networks (CNNs). In AL, one selects the instances to be\nmanually labeled with the aim of maximizing model performance with minimal\neffort. Neural models capitalize on word embeddings as representations\n(features), tuning these to the task at hand. We argue that AL strategies for\nmulti-layered neural models should focus on selecting instances that most\naffect the embedding space (i.e., induce discriminative word representations).\nThis is in contrast to traditional AL approaches (e.g., entropy-based\nuncertainty sampling), which specify higher level objectives. We propose a\nsimple approach for sentence classification that selects instances containing\nwords whose embeddings are likely to be updated with the greatest magnitude,\nthereby rapidly learning discriminative, task-specific embeddings. We extend\nthis approach to document classification by jointly considering: (1) the\nexpected changes to the constituent word representations; and (2) the model's\ncurrent overall uncertainty regarding the instance. The relative emphasis\nplaced on these criteria is governed by a stochastic process that favors\nselecting instances likely to improve representations at the outset of\nlearning, and then shifts toward general uncertainty sampling as AL progresses.\nEmpirical results show that our method outperforms baseline AL approaches on\nboth sentence and document classification tasks. We also show that, as\nexpected, the method quickly learns discriminative word embeddings. To the best\nof our knowledge, this is the first work on AL addressing neural models for\ntext classification.","url_abs":"http://arxiv.org/abs/1606.04212v4","url_pdf":"http://arxiv.org/pdf/1606.04212v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"active-discriminative-text-representation","repo_url":"https://github.com/webis-de/small-text","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"active-learning","task_name":"Active Learning"},{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"document-classification","task_name":"Document Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-classification","task_name":"Sentence Classification"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1606.04212","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}