{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/manifold-characteristics-that-predict","title":"Manifold Characteristics That Predict Downstream Task Performance","arxiv_id":"2205.07477","date":"2022-05-16","proceeding":null,"authors":["Ruan van der Merwe","Gregory Newman","Etienne Barnard"],"abstract":"Pretraining methods are typically compared by evaluating the accuracy of linear classifiers, transfer learning performance, or visually inspecting the representation manifold's (RM) lower-dimensional projections. We show that the differences between methods can be understood more clearly by investigating the RM directly, which allows for a more detailed comparison. To this end, we propose a framework and new metric to measure and compare different RMs. We also investigate and report on the RM characteristics for various pretraining methods. These characteristics are measured by applying sequentially larger local alterations to the input data, using white noise injections and Projected Gradient Descent (PGD) adversarial attacks, and then tracking each datapoint. We calculate the total distance moved for each datapoint and the relative change in distance between successive alterations. We show that self-supervised methods learn an RM where alterations lead to large but constant size changes, indicating a smoother RM than fully supervised methods. We then combine these measurements into one metric, the Representation Manifold Quality Metric (RMQM), where larger values indicate larger and less variable step sizes, and show that RMQM correlates positively with performance on downstream tasks.","url_abs":"https://arxiv.org/abs/2205.07477v1","url_pdf":"https://arxiv.org/pdf/2205.07477v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"manifold-characteristics-that-predict","repo_url":"https://github.com/bytefuse/representation-manifold-quality-metric","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2205.07477","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.07477"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bytefuse/representation-manifold-quality-metric","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"94af383ec267689f","entry":"CifarResNet18","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"src/models.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/src/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94af383ec267689f"}},{"code_sha256_prefix":"3fe98353141c24d9","entry":"calculate_centroid_distances_metrics","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"src/metrics.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/src/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3fe98353141c24d9"}},{"code_sha256_prefix":"47ede95b96e39a52","entry":"calculate_pointwise_distances_metrics","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"src/metrics.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/src/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"47ede95b96e39a52"}},{"code_sha256_prefix":"ab7cb2f6ec7f6e67","entry":"flatten_dict","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab7cb2f6ec7f6e67"}},{"code_sha256_prefix":"12f524933b94fef1","entry":"plot_batch_of_images","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"12f524933b94fef1"}},{"code_sha256_prefix":"f4989573d59e7b36","entry":"quick_fine_tune","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"apply_vector_search.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/apply_vector_search.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f4989573d59e7b36"}},{"code_sha256_prefix":"f5c4d946ceca7d0f","entry":"return_distances_label_wise","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"src/metrics.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/src/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f5c4d946ceca7d0f"}},{"code_sha256_prefix":"495bd3a5caf4e261","entry":"return_fgsm_contrastive_attack_images","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"src/attacks.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/src/attacks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"495bd3a5caf4e261"}},{"code_sha256_prefix":"ffdae627a660a343","entry":"return_fgsm_supervised_attack_images","repo":"bytefuse/representation-manifold-quality-metric","repo_kind":"official","path":"src/attacks.py","file_url":"https://github.com/bytefuse/representation-manifold-quality-metric/blob/HEAD/src/attacks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ffdae627a660a343"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}