{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-empirical-study-into-clustering-of-unseen","title":"An Empirical Study into Clustering of Unseen Datasets with Self-Supervised Encoders","arxiv_id":"2406.02465","date":"2024-06-04","proceeding":null,"authors":["Scott C. Lowe","Joakim Bruslund Haurum","Sageev Oore","Thomas B. Moeslund","Graham W. Taylor"],"abstract":"Can pretrained models generalize to new datasets without any retraining? We deploy pretrained image models on datasets they were not trained for, and investigate whether their embeddings form meaningful clusters. Our suite of benchmarking experiments use encoders pretrained solely on ImageNet-1k with either supervised or self-supervised training techniques, deployed on image datasets that were not seen during training, and clustered with conventional clustering algorithms. This evaluation provides new insights into the embeddings of self-supervised models, which prioritize different features to supervised models. Supervised encoders typically offer more utility than SSL encoders within the training domain, and vice-versa far outside of it, however, fine-tuned encoders demonstrate the opposite trend. Clustering provides a way to evaluate the utility of self-supervised learned representations orthogonal to existing methods such as kNN. Additionally, we find the silhouette score when measured in a UMAP-reduced space is highly correlated with clustering performance, and can therefore be used as a proxy for clustering performance on data with no ground truth labels. Our code implementation is available at \\url{https://github.com/scottclowe/zs-ssl-clustering/}.","url_abs":"https://arxiv.org/abs/2406.02465v1","url_pdf":"https://arxiv.org/pdf/2406.02465v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-empirical-study-into-clustering-of-unseen","repo_url":"https://github.com/scottclowe/zs-ssl-clustering","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"an-empirical-study-into-clustering-of-unseen","repo_url":"https://github.com/bioscan-ml/bioscan-5m","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"an-empirical-study-into-clustering-of-unseen","repo_url":"https://github.com/zahrag/BIOSCAN-5M","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"clustering","task_name":"Clustering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.02465","atlas_url":"https://app.syntology.ai/?focus=2406.02465","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.02465"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zahrag/BIOSCAN-5M","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bioscan-ml/bioscan-5m","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/scottclowe/zs-ssl-clustering","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":15,"unverified":3},"by_repo_kind":{"official":{"samples":18,"ran":15,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f08f66e56561b459","entry":"build_affinity_matrix","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/louvain.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/louvain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f08f66e56561b459"}},{"code_sha256_prefix":"da25bbcd7de8221b","entry":"determine_epoch_seed","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/utils.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da25bbcd7de8221b"}},{"code_sha256_prefix":"1bf2e9396bc00670","entry":"generate_id","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/utils.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bf2e9396bc00670"}},{"code_sha256_prefix":"d7aa9c6a98fa4fc5","entry":"get_continuous_class_map","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/datasets/nabirds.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/datasets/nabirds.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7aa9c6a98fa4fc5"}},{"code_sha256_prefix":"f3d48f6c34f39530","entry":"get_embeddings_path","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/io.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3d48f6c34f39530"}},{"code_sha256_prefix":"f96c64779258568e","entry":"get_pred_path","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/io.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f96c64779258568e"}},{"code_sha256_prefix":"4ebdf4346a9ed2e7","entry":"get_randsizecrop_transform","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/data_transformations.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/data_transformations.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ebdf4346a9ed2e7"}},{"code_sha256_prefix":"c0cd4e224662e2f7","entry":"get_transform","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/data_transformations.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/data_transformations.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0cd4e224662e2f7"}},{"code_sha256_prefix":"2252d616ba4317bf","entry":"image_dataset_sizes","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/datasets/api.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/datasets/api.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2252d616ba4317bf"}},{"code_sha256_prefix":"d53209ecea508922","entry":"load_class_names","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/datasets/nabirds.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/datasets/nabirds.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d53209ecea508922"}},{"code_sha256_prefix":"02112df93daa57bf","entry":"load_hierarchy","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/datasets/nabirds.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/datasets/nabirds.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"02112df93daa57bf"}},{"code_sha256_prefix":"c9646fb0b16f0e8b","entry":"probe_embedding_shape","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/probe.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c9646fb0b16f0e8b"}},{"code_sha256_prefix":"3a3579dedf3fab37","entry":"sanitize_filename","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/io.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/io.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a3579dedf3fab37"}},{"code_sha256_prefix":"3b2cc66fa66500b3","entry":"scale_lr","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/probe.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b2cc66fa66500b3"}},{"code_sha256_prefix":"506400c268aaa688","entry":"setup_linear_classifiers","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/probe.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/probe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"506400c268aaa688"}},{"code_sha256_prefix":"d0fc1c7854a8ad46","entry":"fetch_dataset","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/datasets/api.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/datasets/api.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d0fc1c7854a8ad46"}},{"code_sha256_prefix":"94ded4fcf407632d","entry":"fetch_image_dataset","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/datasets/api.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/datasets/api.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94ded4fcf407632d"}},{"code_sha256_prefix":"c5b56f5bbb6ef6cd","entry":"init_or_resume_wandb_run","repo":"scottclowe/zs-ssl-clustering","repo_kind":"official","path":"zs_ssl_clustering/utils.py","file_url":"https://github.com/scottclowe/zs-ssl-clustering/blob/HEAD/zs_ssl_clustering/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5b56f5bbb6ef6cd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}