{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generalizing-mlps-with-dropouts-batch","title":"Generalizing MLPs With Dropouts, Batch Normalization, and Skip Connections","arxiv_id":"2108.08186","date":"2021-08-18","proceeding":null,"authors":["Taewoon Kim"],"abstract":"A multilayer perceptron (MLP) is typically made of multiple fully connected layers with nonlinear activation functions. There have been several approaches to make them better (e.g. faster convergence, better convergence limit, etc.). But the researches lack structured ways to test them. We test different MLP architectures by carrying out the experiments on the age and gender datasets. We empirically show that by whitening inputs before every linear layer and adding skip connections, our proposed MLP architecture can result in better performance. Since the whitening process includes dropouts, it can also be used to approximate Bayesian inference. We have open sourced our code, and released models and docker images at https://github.com/tae898/age-gender/","url_abs":"https://arxiv.org/abs/2108.08186v2","url_pdf":"https://arxiv.org/pdf/2108.08186v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generalizing-mlps-with-dropouts-batch","repo_url":"https://github.com/tae898/age-gender","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"age-and-gender-classification","task_name":"Age And Gender Classification"},{"task_slug":"age-estimation","task_name":"Age Estimation"},{"task_slug":"gender-prediction","task_name":"Gender Prediction"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/age-and-gender-classification-on-adience-age","task":"Age And Gender Classification","dataset":"Adience Age","model":"RetinaFace + ArcFace + MLP + IC + Skip connections","rank_in_archive_order":9,"of":16,"metrics":{"Accuracy (5-fold)":"60.86"},"uses_additional_data":true},{"leaderboard":"/sota/age-and-gender-classification-on-adience","task":"Age And Gender Classification","dataset":"Adience Gender","model":"RetinaFace + ArcFace + MLP + Skip connections","rank_in_archive_order":4,"of":10,"metrics":{"Accuracy (5-fold)":"90.66"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}