{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/functional-regularization-for-representation","title":"Functional Regularization for Representation Learning: A Unified Theoretical Perspective","arxiv_id":"2008.02447","date":"2020-08-06","proceeding":"NeurIPS 2020 12","authors":["Siddhant Garg","YIngyu Liang"],"abstract":"Unsupervised and self-supervised learning approaches have become a crucial tool to learn representations for downstream prediction tasks. While these approaches are widely used in practice and achieve impressive empirical gains, their theoretical understanding largely lags behind. Towards bridging this gap, we present a unifying perspective where several such approaches can be viewed as imposing a regularization on the representation via a learnable function using unlabeled data. We propose a discriminative theoretical framework for analyzing the sample complexity of these approaches, which generalizes the framework of (Balcan and Blum, 2010) to allow learnable regularization functions. Our sample complexity bounds show that, with carefully chosen hypothesis classes to exploit the structure in the data, these learnable regularization functions can prune the hypothesis space, and help reduce the amount of labeled data needed. We then provide two concrete examples of functional regularization, one using auto-encoders and the other using masked self-supervision, and apply our framework to quantify the reduction in the sample complexity bound of labeled data. We also provide complementary empirical results to support our analysis.","url_abs":"https://arxiv.org/abs/2008.02447v3","url_pdf":"https://arxiv.org/pdf/2008.02447v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"functional-regularization-for-representation","repo_url":"https://github.com/sid7954/functional-regularization","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"unsupervised-pre-training","task_name":"Unsupervised Pre-training"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2008.02447","atlas_url":"https://app.syntology.ai/?focus=2008.02447","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.02447"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sid7954/functional-regularization","reach":null}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"2beaf56dc129eeba","entry":"F_network","repo":"sid7954/functional-regularization","repo_kind":"official","path":"auto-encoder/func_reg.py","file_url":"https://github.com/sid7954/functional-regularization/blob/HEAD/auto-encoder/func_reg.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2beaf56dc129eeba"}},{"code_sha256_prefix":"912de89a700855ac","entry":"Feedforward","repo":"sid7954/functional-regularization","repo_kind":"official","path":"auto-encoder/func_reg.py","file_url":"https://github.com/sid7954/functional-regularization/blob/HEAD/auto-encoder/func_reg.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"912de89a700855ac"}},{"code_sha256_prefix":"7f7f7dc12d5f39ca","entry":"G_network","repo":"sid7954/functional-regularization","repo_kind":"official","path":"auto-encoder/func_reg.py","file_url":"https://github.com/sid7954/functional-regularization/blob/HEAD/auto-encoder/func_reg.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f7f7dc12d5f39ca"}},{"code_sha256_prefix":"d513ea5c05d9fc9d","entry":"dataloader","repo":"sid7954/functional-regularization","repo_kind":"official","path":"auto-encoder/func_reg.py","file_url":"https://github.com/sid7954/functional-regularization/blob/HEAD/auto-encoder/func_reg.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d513ea5c05d9fc9d"}},{"code_sha256_prefix":"c68fe9ee3926d757","entry":"main","repo":"sid7954/functional-regularization","repo_kind":"official","path":"auto-encoder/func_reg.py","file_url":"https://github.com/sid7954/functional-regularization/blob/HEAD/auto-encoder/func_reg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c68fe9ee3926d757"}},{"code_sha256_prefix":"a39d4a88da6d9dd2","entry":"save","repo":"sid7954/functional-regularization","repo_kind":"official","path":"auto-encoder/func_reg.py","file_url":"https://github.com/sid7954/functional-regularization/blob/HEAD/auto-encoder/func_reg.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a39d4a88da6d9dd2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}