{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/enact-entropy-based-clustering-of-attention","title":"ENACT: Entropy-based Clustering of Attention Input for Improving the Computational Performance of Object Detection Transformers","arxiv_id":"2409.07541","date":"2024-09-11","proceeding":null,"authors":["Giorgos Savathrakis","Antonis Argyros"],"abstract":"Transformers demonstrate competitive performance in terms of precision on the problem of vision-based object detection. However, they require considerable computational resources due to the quadratic size of the attention weights. In this work, we propose to cluster the transformer input on the basis of its entropy. The reason for this is that the self-information of each pixel (whose sum is the entropy), is likely to be similar among pixels corresponding to the same objects. Clustering reduces the size of data given as input to the transformer and therefore reduces training time and GPU memory usage, while at the same time preserves meaningful information to be passed through the remaining parts of the network. The proposed process is organized in a module called ENACT, that can be plugged-in any transformer architecture that consists of a multi-head self-attention computation in its encoder. We ran extensive experiments using the COCO object detection dataset, and three detection transformers. The obtained results demonstrate that in all tested cases, there is consistent reduction in the required computational resources, while the precision of the detection task is only slightly reduced. The code of the ENACT module will become available at https://github.com/GSavathrakis/ENACT","url_abs":"https://arxiv.org/abs/2409.07541v1","url_pdf":"https://arxiv.org/pdf/2409.07541v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"enact-entropy-based-clustering-of-attention","repo_url":"https://github.com/gsavathrakis/enact","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.07541","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.07541"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gsavathrakis/enact","reach":{"status":"ok"}}],"summary":{"ran_violates":1,"ran":4,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"ac8fe530cdad4d8c","entry":"dice_loss","repo":"gsavathrakis/enact","repo_kind":"official","path":"Anchor-DETR-ENACT/models/segmentation.py","file_url":"https://github.com/gsavathrakis/enact/blob/HEAD/Anchor-DETR-ENACT/models/segmentation.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac8fe530cdad4d8c"}},{"code_sha256_prefix":"96700f4fbea456b4","entry":"fmt_res","repo":"gsavathrakis/enact","repo_kind":"official","path":"Anchor-DETR-ENACT/compute_speeds.py","file_url":"https://github.com/gsavathrakis/enact/blob/HEAD/Anchor-DETR-ENACT/compute_speeds.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"96700f4fbea456b4"}},{"code_sha256_prefix":"60bb6b80d631b409","entry":"measure_time","repo":"gsavathrakis/enact","repo_kind":"official","path":"Anchor-DETR-ENACT/compute_speeds.py","file_url":"https://github.com/gsavathrakis/enact/blob/HEAD/Anchor-DETR-ENACT/compute_speeds.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"60bb6b80d631b409"}},{"code_sha256_prefix":"b4a2dd35394d50d6","entry":"pos2posemb1d","repo":"gsavathrakis/enact","repo_kind":"official","path":"Anchor-DETR-ENACT/models/transformer.py","file_url":"https://github.com/gsavathrakis/enact/blob/HEAD/Anchor-DETR-ENACT/models/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b4a2dd35394d50d6"}},{"code_sha256_prefix":"b7f23e7c2f412bd5","entry":"pos2posemb2d","repo":"gsavathrakis/enact","repo_kind":"official","path":"Anchor-DETR-ENACT/models/transformer.py","file_url":"https://github.com/gsavathrakis/enact/blob/HEAD/Anchor-DETR-ENACT/models/transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b7f23e7c2f412bd5"}},{"code_sha256_prefix":"5c0711aada67957e","entry":"sigmoid_focal_loss","repo":"gsavathrakis/enact","repo_kind":"official","path":"Anchor-DETR-ENACT/models/segmentation.py","file_url":"https://github.com/gsavathrakis/enact/blob/HEAD/Anchor-DETR-ENACT/models/segmentation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c0711aada67957e"}},{"code_sha256_prefix":"e6ecb133c6711c02","entry":"multi_head_rcda_forward_dec","repo":"gsavathrakis/enact","repo_kind":"official","path":"Anchor-DETR-ENACT/models/row_column_decoupled_attention.py","file_url":"https://github.com/gsavathrakis/enact/blob/HEAD/Anchor-DETR-ENACT/models/row_column_decoupled_attention.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e6ecb133c6711c02"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}