{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/u2-former-a-nested-u-shaped-transformer-for","title":"U2-Former: A Nested U-shaped Transformer for Image Restoration","arxiv_id":"2112.02279","date":"2021-12-04","proceeding":null,"authors":["Haobo Ji","Xin Feng","Wenjie Pei","Jinxing Li","Guangming Lu"],"abstract":"While Transformer has achieved remarkable performance in various high-level vision tasks, it is still challenging to exploit the full potential of Transformer in image restoration. The crux lies in the limited depth of applying Transformer in the typical encoder-decoder framework for image restoration, resulting from heavy self-attention computation load and inefficient communications across different depth (scales) of layers. In this paper, we present a deep and effective Transformer-based network for image restoration, termed as U2-Former, which is able to employ Transformer as the core operation to perform image restoration in a deep encoding and decoding space. Specifically, it leverages the nested U-shaped structure to facilitate the interactions across different layers with different scales of feature maps. Furthermore, we optimize the computational efficiency for the basic Transformer block by introducing a feature-filtering mechanism to compress the token representation. Apart from the typical supervision ways for image restoration, our U2-Former also performs contrastive learning in multiple aspects to further decouple the noise component from the background image. Extensive experiments on various image restoration tasks, including reflection removal, rain streak removal and dehazing respectively, demonstrate the effectiveness of the proposed U2-Former.","url_abs":"https://arxiv.org/abs/2112.02279v2","url_pdf":"https://arxiv.org/pdf/2112.02279v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"image-dehazing","task_name":"Image Dehazing"},{"task_slug":"image-restoration","task_name":"Image Restoration"},{"task_slug":"reflection-removal","task_name":"Reflection Removal"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-dehazing-on-sots-indoor","task":"Image Dehazing","dataset":"SOTS Indoor","model":"U2-Former","rank_in_archive_order":21,"of":34,"metrics":{"PSNR":"36.42","SSIM":"0.988"},"uses_additional_data":false},{"leaderboard":"/sota/image-dehazing-on-sots-outdoor","task":"Image Dehazing","dataset":"SOTS Outdoor","model":"U2-Former","rank_in_archive_order":22,"of":31,"metrics":{"PSNR":"31.10","SSIM":"0.976"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.02279","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}