{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/strengthening-layer-interaction-via-dynamic","title":"Strengthening Layer Interaction via Dynamic Layer Attention","arxiv_id":"2406.13392","date":"2024-06-19","proceeding":null,"authors":["Kaishen Wang","Xun Xia","Jian Liu","Zhang Yi","Tao He"],"abstract":"In recent years, employing layer attention to enhance interaction among hierarchical layers has proven to be a significant advancement in building network structures. In this paper, we delve into the distinction between layer attention and the general attention mechanism, noting that existing layer attention methods achieve layer interaction on fixed feature maps in a static manner. These static layer attention methods limit the ability for context feature extraction among layers. To restore the dynamic context representation capability of the attention mechanism, we propose a Dynamic Layer Attention (DLA) architecture. The DLA comprises dual paths, where the forward path utilizes an improved recurrent neural network block, named Dynamic Sharing Unit (DSU), for context feature extraction. The backward path updates features using these shared context representations. Finally, the attention mechanism is applied to these dynamically refreshed feature maps among layers. Experimental results demonstrate the effectiveness of the proposed DLA architecture, outperforming other state-of-the-art methods in image recognition and object detection tasks. Additionally, the DSU block has been evaluated as an efficient plugin in the proposed DLA architecture.The code is available at https://github.com/tunantu/Dynamic-Layer-Attention.","url_abs":"https://arxiv.org/abs/2406.13392v1","url_pdf":"https://arxiv.org/pdf/2406.13392v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"strengthening-layer-interaction-via-dynamic","repo_url":"https://github.com/tunantu/dynamic-layer-attention","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dla","method_name":"DLA"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.13392","atlas_url":"https://app.syntology.ai/?focus=2406.13392","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.13392"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tunantu/dynamic-layer-attention","reach":{"status":"ok"}}],"summary":{"ran_fixture":2,"ran":7,"ran_draft_wrong":4,"unverified":1},"by_repo_kind":{"official":{"samples":14,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"131a82fd65128218","entry":"accuracy","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/train_pro.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/train_pro.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"131a82fd65128218"}},{"code_sha256_prefix":"9c2113a7471b6758","entry":"clear_log","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/best.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/best.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9c2113a7471b6758"}},{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/models/resnet.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/models/resnet.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/models/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/cifar/models/cifar/dia_resnet.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/cifar/models/cifar/dia_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"dd1114865f06f0fd","entry":"conv3x3","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/cifar/models/cifar/dla_b.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/cifar/models/cifar/dla_b.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dd1114865f06f0fd"}},{"code_sha256_prefix":"39eace7e2822504f","entry":"drop_path","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/cifar/models/cifar/dla_b.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/cifar/models/cifar/dla_b.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"39eace7e2822504f"}},{"code_sha256_prefix":"be91a77e5ce083f4","entry":"read_acclog","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/best.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/best.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"be91a77e5ce083f4"}},{"code_sha256_prefix":"8f4c7baa0e1369a6","entry":"read_acclog","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/eval_visual.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/eval_visual.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8f4c7baa0e1369a6"}},{"code_sha256_prefix":"6afb73a5970361f7","entry":"read_losslog","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/best.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/best.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6afb73a5970361f7"}},{"code_sha256_prefix":"892daf8a1afd63ed","entry":"read_losslog","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/eval_visual.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/eval_visual.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"892daf8a1afd63ed"}},{"code_sha256_prefix":"d6a70da7dc77eebd","entry":"train","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/train_pro.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/train_pro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d6a70da7dc77eebd"}},{"code_sha256_prefix":"724424f5001337b0","entry":"validate","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/train_pro.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/train_pro.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"724424f5001337b0"}},{"code_sha256_prefix":"852fb4356af43925","entry":"resnet50_eca","repo":"tunantu/dynamic-layer-attention","repo_kind":"official","path":"image_classification/imagenet/models/resnet.py","file_url":"https://github.com/tunantu/dynamic-layer-attention/blob/HEAD/image_classification/imagenet/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"852fb4356af43925"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}