{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/k-sparse-autoencoders","title":"k-Sparse Autoencoders","arxiv_id":"1312.5663","date":"2013-12-19","proceeding":null,"authors":["Alireza Makhzani","Brendan Frey"],"abstract":"Recently, it has been observed that when representations are learnt in a way\nthat encourages sparsity, improved performance is obtained on classification\ntasks. These methods involve combinations of activation functions, sampling\nsteps and different kinds of penalties. To investigate the effectiveness of\nsparsity by itself, we propose the k-sparse autoencoder, which is an\nautoencoder with linear activation function, where in hidden layers only the k\nhighest activities are kept. When applied to the MNIST and NORB datasets, we\nfind that this method achieves better classification results than denoising\nautoencoders, networks trained with dropout, and RBMs. k-sparse autoencoders\nare simple to train and the encoding stage is very fast, making them\nwell-suited to large problem sizes, where conventional sparse coding algorithms\ncannot be applied.","url_abs":"http://arxiv.org/abs/1312.5663v2","url_pdf":"http://arxiv.org/pdf/1312.5663v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"k-sparse-autoencoders","repo_url":"https://github.com/Sharda-Borse/AutoEncoders","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}},{"paper_slug":"k-sparse-autoencoders","repo_url":"https://github.com/kyeongry/Autoencoders","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"k-sparse-autoencoders","repo_url":"https://github.com/windowtodata/python-projects","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"classification","task_name":"General Classification"}],"methods":[{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"k-sparse-autoencoder","method_name":"k-Sparse Autoencoder"}],"datasets_introduced":[],"methods_introduced":[{"slug":"k-sparse-autoencoder","name":"k-Sparse Autoencoder","full_name":"k-Sparse Autoencoder"}],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1312.5663","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}