{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diet-networks-thin-parameters-for-fat","title":"Diet Networks: Thin Parameters for Fat Genomics","arxiv_id":"1611.09340","date":"2016-11-28","proceeding":null,"authors":["Adriana Romero","Pierre Luc Carrier","Akram Erraqabi","Tristan Sylvain","Alex Auvolat","Etienne Dejoie","Marc-André Legault","Marie-Pierre Dubé","Julie G. Hussin","Yoshua Bengio"],"abstract":"Learning tasks such as those involving genomic data often poses a serious\nchallenge: the number of input features can be orders of magnitude larger than\nthe number of training examples, making it difficult to avoid overfitting, even\nwhen using the known regularization techniques. We focus here on tasks in which\nthe input is a description of the genetic variation specific to a patient, the\nsingle nucleotide polymorphisms (SNPs), yielding millions of ternary inputs.\nImproving the ability of deep learning to handle such datasets could have an\nimportant impact in precision medicine, where high-dimensional data regarding a\nparticular patient is used to make predictions of interest. Even though the\namount of data for such tasks is increasing, this mismatch between the number\nof examples and the number of inputs remains a concern. Naive implementations\nof classifier neural networks involve a huge number of free parameters in their\nfirst layer: each input feature is associated with as many parameters as there\nare hidden units. We propose a novel neural network parametrization which\nconsiderably reduces the number of free parameters. It is based on the idea\nthat we can first learn or provide a distributed representation for each input\nfeature (e.g. for each position in the genome where variations are observed),\nand then learn (with another neural network called the parameter prediction\nnetwork) how to map a feature's distributed representation to the vector of\nparameters specific to that feature in the classifier neural network (the\nweights which link the value of the feature to each of the hidden units). We\nshow experimentally on a population stratification task of interest to medical\nstudies that the proposed approach can significantly reduce both the number of\nparameters and the error rate of the classifier.","url_abs":"http://arxiv.org/abs/1611.09340v3","url_pdf":"http://arxiv.org/pdf/1611.09340v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diet-networks-thin-parameters-for-fat","repo_url":"https://github.com/adri-romsor/DietNetworks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"diet-networks-thin-parameters-for-fat","repo_url":"https://github.com/alexandremarcil/DietNetworks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"diet-networks-thin-parameters-for-fat","repo_url":"https://github.com/rochefca/DIETNETWORK","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}},{"paper_slug":"diet-networks-thin-parameters-for-fat","repo_url":"https://github.com/ze1gades/diary","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"diet-networks-thin-parameters-for-fat","repo_url":"https://github.com/miritrope/genome","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"parameter-prediction","task_name":"Parameter Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1611.09340","atlas_url":"https://app.syntology.ai/?focus=1611.09340","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1611.09340"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/adri-romsor/DietNetworks","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alexandremarcil/DietNetworks","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ze1gades/diary","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rochefca/DIETNETWORK","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/miritrope/genome","reach":{"status":"ok"}}],"summary":{"ran_honours":2,"unverified":1},"by_repo_kind":{"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"fadad5af287bb7e6","entry":"test_pass_diet","repo":"alexandremarcil/DietNetworks","repo_kind":"listed","path":"training.py","file_url":"https://github.com/alexandremarcil/DietNetworks/blob/HEAD/training.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fadad5af287bb7e6"}},{"code_sha256_prefix":"7807aa88e95d70c4","entry":"train_pass_diet","repo":"alexandremarcil/DietNetworks","repo_kind":"listed","path":"training.py","file_url":"https://github.com/alexandremarcil/DietNetworks/blob/HEAD/training.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7807aa88e95d70c4"}},{"code_sha256_prefix":"2b7bfc6d814112d5","entry":"run_training_diet","repo":"alexandremarcil/DietNetworks","repo_kind":"listed","path":"training.py","file_url":"https://github.com/alexandremarcil/DietNetworks/blob/HEAD/training.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b7bfc6d814112d5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}