{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-identity-mappings-with-residual","title":"Learning Identity Mappings with Residual Gates","arxiv_id":"1611.01260","date":"2016-11-04","proceeding":null,"authors":["Pedro H. P. Savarese","Leonardo O. Mazza","Daniel R. Figueiredo"],"abstract":"We propose a new layer design by adding a linear gating mechanism to shortcut\nconnections. By using a scalar parameter to control each gate, we provide a way\nto learn identity mappings by optimizing only one parameter. We build upon the\nmotivation behind Residual Networks, where a layer is reformulated in order to\nmake learning identity mappings less problematic to the optimizer. The\naugmentation introduces only one extra parameter per layer, and provides easier\noptimization by making degeneration into identity mappings simpler. We propose\na new model, the Gated Residual Network, which is the result when augmenting\nResidual Networks. Experimental results show that augmenting layers provides\nbetter optimization, increased performance, and more layer independence. We\nevaluate our method on MNIST using fully-connected networks, showing empirical\nindications that our augmentation facilitates the optimization of deep models,\nand that it provides high tolerance to full layer removal: the model retains\nover 90% of its performance even after half of its layers have been randomly\nremoved. We also evaluate our model on CIFAR-10 and CIFAR-100 using Wide Gated\nResNets, achieving 3.65% and 18.27% error, respectively.","url_abs":"http://arxiv.org/abs/1611.01260v2","url_pdf":"http://arxiv.org/pdf/1611.01260v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"Residual Gates + WRN","rank_in_archive_order":116,"of":265,"metrics":{"Percentage correct":"96.35"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-cifar-100","task":"Image Classification","dataset":"CIFAR-100","model":"Residual Gates + WRN","rank_in_archive_order":114,"of":211,"metrics":{"Percentage correct":"81.73"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.01260","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}