{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/training-objective-drives-the-consistency-of","title":"Training objective drives the consistency of representational similarity across datasets","arxiv_id":"2411.05561","date":"2024-11-08","proceeding":null,"authors":["Laure Ciernik","Lorenz Linhardt","Marco Morik","Jonas Dippel","Simon Kornblith","Lukas Muttenthaler"],"abstract":"The Platonic Representation Hypothesis claims that recent foundation models are converging to a shared representation space as a function of their downstream task performance, irrespective of the objectives and data modalities used to train these models. Representational similarity is generally measured for individual datasets and is not necessarily consistent across datasets. Thus, one may wonder whether this convergence of model representations is confounded by the datasets commonly used in machine learning. Here, we propose a systematic way to measure how representational similarity between models varies with the set of stimuli used to construct the representations. We find that the objective function is the most crucial factor in determining the consistency of representational similarities across datasets. Specifically, self-supervised vision models learn representations whose relative pairwise similarities generalize better from one dataset to another compared to those of image classification or image-text models. Moreover, the correspondence between representational similarities and the models' task behavior is dataset-dependent, being most strongly pronounced for single-domain datasets. Our work provides a framework for systematically measuring similarities of model representations across datasets and linking those similarities to differences in task behavior.","url_abs":"https://arxiv.org/abs/2411.05561v1","url_pdf":"https://arxiv.org/pdf/2411.05561v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"training-objective-drives-the-consistency-of","repo_url":"https://github.com/lciernik/similarity_consistency","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.05561","atlas_url":"https://app.syntology.ai/?focus=2411.05561","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.05561"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lciernik/similarity_consistency","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"67833bbbacc8895a","entry":"accuracy","repo":"lciernik/similarity_consistency","repo_kind":"official","path":"sim_consistency/eval/metrics.py","file_url":"https://github.com/lciernik/similarity_consistency/blob/HEAD/sim_consistency/eval/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67833bbbacc8895a"}},{"code_sha256_prefix":"0bc64cda18d42ef8","entry":"check_paths","repo":"lciernik/similarity_consistency","repo_kind":"official","path":"sim_consistency/cluster_models.py","file_url":"https://github.com/lciernik/similarity_consistency/blob/HEAD/sim_consistency/cluster_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0bc64cda18d42ef8"}},{"code_sha256_prefix":"6beb4e86a582c341","entry":"compute_metrics","repo":"lciernik/similarity_consistency","repo_kind":"official","path":"sim_consistency/eval/metrics.py","file_url":"https://github.com/lciernik/similarity_consistency/blob/HEAD/sim_consistency/eval/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6beb4e86a582c341"}},{"code_sha256_prefix":"76a740248191180d","entry":"load_similarity_matrix","repo":"lciernik/similarity_consistency","repo_kind":"official","path":"sim_consistency/cluster_models.py","file_url":"https://github.com/lciernik/similarity_consistency/blob/HEAD/sim_consistency/cluster_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76a740248191180d"}},{"code_sha256_prefix":"bf3f3101aee181de","entry":"prepare_args","repo":"lciernik/similarity_consistency","repo_kind":"official","path":"sim_consistency/argparser.py","file_url":"https://github.com/lciernik/similarity_consistency/blob/HEAD/sim_consistency/argparser.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf3f3101aee181de"}},{"code_sha256_prefix":"cf4aa653f8b09878","entry":"prepare_combined_args","repo":"lciernik/similarity_consistency","repo_kind":"official","path":"sim_consistency/argparser.py","file_url":"https://github.com/lciernik/similarity_consistency/blob/HEAD/sim_consistency/argparser.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf4aa653f8b09878"}},{"code_sha256_prefix":"5efe179eb12a6674","entry":"process_similarity_matrix","repo":"lciernik/similarity_consistency","repo_kind":"official","path":"sim_consistency/cluster_models.py","file_url":"https://github.com/lciernik/similarity_consistency/blob/HEAD/sim_consistency/cluster_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5efe179eb12a6674"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}