{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/approximation-vector-machines-for-large-scale","title":"Approximation Vector Machines for Large-scale Online Learning","arxiv_id":"1604.06518","date":"2016-04-22","proceeding":null,"authors":["Trung Le","Tu Dinh Nguyen","Vu Nguyen","Dinh Phung"],"abstract":"One of the most challenging problems in kernel online learning is to bound\nthe model size and to promote the model sparsity. Sparse models not only\nimprove computation and memory usage, but also enhance the generalization\ncapacity, a principle that concurs with the law of parsimony. However,\ninappropriate sparsity modeling may also significantly degrade the performance.\nIn this paper, we propose Approximation Vector Machine (AVM), a model that can\nsimultaneously encourage the sparsity and safeguard its risk in compromising\nthe performance. When an incoming instance arrives, we approximate this\ninstance by one of its neighbors whose distance to it is less than a predefined\nthreshold. Our key intuition is that since the newly seen instance is expressed\nby its nearby neighbor the optimal performance can be analytically formulated\nand maintained. We develop theoretical foundations to support this intuition\nand further establish an analysis to characterize the gap between the\napproximation and optimal solutions. This gap crucially depends on the\nfrequency of approximation and the predefined threshold. We perform the\nconvergence analysis for a wide spectrum of loss functions including Hinge,\nsmooth Hinge, and Logistic for classification task, and $l_1$, $l_2$, and\n$\\epsilon$-insensitive for regression task. We conducted extensive experiments\nfor classification task in batch and online modes, and regression task in\nonline mode over several benchmark datasets. The results show that our proposed\nAVM achieved a comparable predictive performance with current state-of-the-art\nmethods while simultaneously achieving significant computational speed-up due\nto the ability of the proposed AVM in maintaining the model size.","url_abs":"http://arxiv.org/abs/1604.06518v4","url_pdf":"http://arxiv.org/pdf/1604.06518v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"approximation-vector-machines-for-large-scale","repo_url":"https://github.com/tund/avm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}