{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-networks-regularization-through","title":"Neural Networks Regularization Through Representation Learning","arxiv_id":"1807.05292","date":"2018-07-13","proceeding":null,"authors":["Soufiane Belharbi"],"abstract":"Neural network models and deep models are one of the leading and state of the\nart models in machine learning. Most successful deep neural models are the ones\nwith many layers which highly increases their number of parameters. Training\nsuch models requires a large number of training samples which is not always\navailable. One of the fundamental issues in neural networks is overfitting\nwhich is the issue tackled in this thesis. Such problem often occurs when the\ntraining of large models is performed using few training samples. Many\napproaches have been proposed to prevent the network from overfitting and\nimprove its generalization performance such as data augmentation, early\nstopping, parameters sharing, unsupervised learning, dropout, batch\nnormalization, etc.\n  In this thesis, we tackle the neural network overfitting issue from a\nrepresentation learning perspective by considering the situation where few\ntraining samples are available which is the case of many real world\napplications. We propose three contributions. The first one presented in\nchapter 2 is dedicated to dealing with structured output problems to perform\nmultivariate regression when the output variable y contains structural\ndependencies between its components. The second contribution described in\nchapter 3 deals with the classification task where we propose to exploit prior\nknowledge about the internal representation of the hidden layers in neural\nnetworks. Our last contribution presented in chapter 4 showed the interest of\ntransfer learning in applications where only few samples are available. In this\ncontribution, we provide an automatic system based on such learning scheme with\nan application to medical domain. In this application, the task consists in\nlocalizing the third lumbar vertebra in a 3D CT scan. This work has been done\nin collaboration with the clinic Rouen Henri Becquerel Center who provided us\nwith data.","url_abs":"http://arxiv.org/abs/1807.05292v1","url_pdf":"http://arxiv.org/pdf/1807.05292v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-networks-regularization-through","repo_url":"https://github.com/sbelharbi/learning-class-invariant-features","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}