{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/segmentation-aware-convolutional-networks","title":"Segmentation-Aware Convolutional Networks Using Local Attention Masks","arxiv_id":"1708.04607","date":"2017-08-15","proceeding":"ICCV 2017 10","authors":["Adam W. Harley","Konstantinos G. Derpanis","Iasonas Kokkinos"],"abstract":"We introduce an approach to integrate segmentation information within a\nconvolutional neural network (CNN). This counter-acts the tendency of CNNs to\nsmooth information across regions and increases their spatial precision. To\nobtain segmentation information, we set up a CNN to provide an embedding space\nwhere region co-membership can be estimated based on Euclidean distance. We use\nthese embeddings to compute a local attention mask relative to every neuron\nposition. We incorporate such masks in CNNs and replace the convolution\noperation with a \"segmentation-aware\" variant that allows a neuron to\nselectively attend to inputs coming from its own region. We call the resulting\nnetwork a segmentation-aware CNN because it adapts its filters at each image\npoint according to local segmentation cues. We demonstrate the merit of our\nmethod on two widely different dense prediction tasks, that involve\nclassification (semantic segmentation) and regression (optical flow). Our\nresults show that in semantic segmentation we can match the performance of\nDenseCRFs while being faster and simpler, and in optical flow we obtain clearly\nsharper responses than networks that do not use local attention masks. In both\ncases, segmentation-aware convolution yields systematic improvements over\nstrong baselines. Source code for this work is available online at\nhttp://cs.cmu.edu/~aharley/segaware.","url_abs":"http://arxiv.org/abs/1708.04607v1","url_pdf":"http://arxiv.org/pdf/1708.04607v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"segmentation-aware-convolutional-networks","repo_url":"https://github.com/aharley/segaware","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1708.04607","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}