{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/regularizing-model-complexity-and-label","title":"Regularizing Model Complexity and Label Structure for Multi-Label Text Classification","arxiv_id":"1705.00740","date":"2017-05-01","proceeding":null,"authors":["Bingyu Wang","Cheng Li","Virgil Pavlu","Javed Aslam"],"abstract":"Multi-label text classification is a popular machine learning task where each\ndocument is assigned with multiple relevant labels. This task is challenging\ndue to high dimensional features and correlated labels. Multi-label text\nclassifiers need to be carefully regularized to prevent the severe over-fitting\nin the high dimensional space, and also need to take into account label\ndependencies in order to make accurate predictions under uncertainty. We\ndemonstrate significant and practical improvement by carefully regularizing the\nmodel complexity during training phase, and also regularizing the label search\nspace during prediction phase. Specifically, we regularize the classifier\ntraining using Elastic-net (L1+L2) penalty for reducing model complexity/size,\nand employ early stopping to prevent overfitting. At prediction time, we apply\nsupport inference to restrict the search space to label sets encountered in the\ntraining set, and F-optimizer GFM to make optimal predictions for the F1\nmetric. We show that although support inference only provides density\nestimations on existing label combinations, when combined with GFM predictor,\nthe algorithm can output unseen label combinations. Taken collectively, our\nexperiments show state of the art results on many benchmark datasets. Beyond\nperformance and practical contributions, we make some interesting observations.\nContrary to the prior belief, which deems support inference as purely an\napproximate inference procedure, we show that support inference acts as a\nstrong regularizer on the label prediction structure. It allows the classifier\nto take into account label dependencies during prediction even if the\nclassifiers had not modeled any label dependencies during training.","url_abs":"http://arxiv.org/abs/1705.00740v1","url_pdf":"http://arxiv.org/pdf/1705.00740v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"regularizing-model-complexity-and-label","repo_url":"https://github.com/cheng-li/pyramid","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"multi-label-text-classification-1","task_name":"Multi Label Text Classification"},{"task_slug":"multi-label-text-classification","task_name":"Multi-Label Text Classification"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"early-stopping","method_name":"Early Stopping"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1705.00740","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}