{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/neural-clustering-based-visual-representation","title":"Neural Clustering based Visual Representation Learning","arxiv_id":"2403.17409","date":"2024-03-26","proceeding":"CVPR 2024 1","authors":["Guikun Chen","Xia Li","Yi Yang","Wenguan Wang"],"abstract":"We investigate a fundamental aspect of machine vision: the measurement of features, by revisiting clustering, one of the most classic approaches in machine learning and data analysis. Existing visual feature extractors, including ConvNets, ViTs, and MLPs, represent an image as rectangular regions. Though prevalent, such a grid-style paradigm is built upon engineering practice and lacks explicit modeling of data distribution. In this work, we propose feature extraction with clustering (FEC), a conceptually elegant yet surprisingly ad-hoc interpretable neural clustering framework, which views feature extraction as a process of selecting representatives from data and thus automatically captures the underlying data distribution. Given an image, FEC alternates between grouping pixels into individual clusters to abstract representatives and updating the deep features of pixels with current representatives. Such an iterative working mechanism is implemented in the form of several neural layers and the final representatives can be used for downstream tasks. The cluster assignments across layers, which can be viewed and inspected by humans, make the forward process of FEC fully transparent and empower it with promising ad-hoc interpretability. Extensive experiments on various visual recognition models and tasks verify the effectiveness, generality, and interpretability of FEC. We expect this work will provoke a rethink of the current de facto grid-style paradigm.","url_abs":"https://arxiv.org/abs/2403.17409v1","url_pdf":"https://arxiv.org/pdf/2403.17409v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"neural-clustering-based-visual-representation","repo_url":"https://github.com/guikunchen/fec","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2403.17409","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.17409"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/guikunchen/fec","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"81660a718d852b93","entry":"aggregate_masks","repo":"guikunchen/fec","repo_kind":"official","path":"cluster_visualize.py","file_url":"https://github.com/guikunchen/fec/blob/HEAD/cluster_visualize.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"81660a718d852b93"}},{"code_sha256_prefix":"aa8d49041d9f8442","entry":"get_num_layer_layer_wise","repo":"guikunchen/fec","repo_kind":"official","path":"detection/mmcv_custom/layer_decay_optimizer_constructor.py","file_url":"https://github.com/guikunchen/fec/blob/HEAD/detection/mmcv_custom/layer_decay_optimizer_constructor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa8d49041d9f8442"}},{"code_sha256_prefix":"be134d6ef63b5192","entry":"get_num_layer_stage_wise","repo":"guikunchen/fec","repo_kind":"official","path":"detection/mmcv_custom/layer_decay_optimizer_constructor.py","file_url":"https://github.com/guikunchen/fec/blob/HEAD/detection/mmcv_custom/layer_decay_optimizer_constructor.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"be134d6ef63b5192"}},{"code_sha256_prefix":"548053a14e52b92d","entry":"pairwise_cos_sim","repo":"guikunchen/fec","repo_kind":"official","path":"cluster_visualize.py","file_url":"https://github.com/guikunchen/fec/blob/HEAD/cluster_visualize.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"548053a14e52b92d"}},{"code_sha256_prefix":"db1ef09456179a68","entry":"pairwise_cos_sim","repo":"guikunchen/fec","repo_kind":"official","path":"models/fec.py","file_url":"https://github.com/guikunchen/fec/blob/HEAD/models/fec.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"db1ef09456179a68"}},{"code_sha256_prefix":"0fc8219dc215c3b1","entry":"validate","repo":"guikunchen/fec","repo_kind":"official","path":"validate.py","file_url":"https://github.com/guikunchen/fec/blob/HEAD/validate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0fc8219dc215c3b1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}