{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mano-exploiting-matrix-norm-for-unsupervised","title":"MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts","arxiv_id":"2405.18979","date":"2024-05-29","proceeding":null,"authors":["Renchunzi Xie","Ambroise Odonnat","Vasilii Feofanov","Weijian Deng","Jianfeng Zhang","Bo An"],"abstract":"Leveraging the models' outputs, specifically the logits, is a common approach to estimating the test accuracy of a pre-trained neural network on out-of-distribution (OOD) samples without requiring access to the corresponding ground truth labels. Despite their ease of implementation and computational efficiency, current logit-based methods are vulnerable to overconfidence issues, leading to prediction bias, especially under the natural shift. In this work, we first study the relationship between logits and generalization performance from the view of low-density separation assumption. Our findings motivate our proposed method MaNo which (1) applies a data-dependent normalization on the logits to reduce prediction bias, and (2) takes the $L_p$ norm of the matrix of normalized logits as the estimation score. Our theoretical analysis highlights the connection between the provided score and the model's uncertainty. We conduct an extensive empirical study on common unsupervised accuracy estimation benchmarks and demonstrate that MaNo achieves state-of-the-art performance across various architectures in the presence of synthetic, natural, or subpopulation shifts. The code is available at \\url{https://github.com/Renchunzi-Xie/MaNo}.","url_abs":"https://arxiv.org/abs/2405.18979v3","url_pdf":"https://arxiv.org/pdf/2405.18979v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mano-exploiting-matrix-norm-for-unsupervised","repo_url":"https://github.com/renchunzi-xie/mano","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.18979","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.18979"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/renchunzi-xie/mano","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c4894fa08bdadbde","entry":"ResNet18","repo":"renchunzi-xie/mano","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/renchunzi-xie/mano/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c4894fa08bdadbde"}},{"code_sha256_prefix":"5ca5fed85b4fc975","entry":"ResNet50","repo":"renchunzi-xie/mano","repo_kind":"official","path":"models/resnet.py","file_url":"https://github.com/renchunzi-xie/mano/blob/HEAD/models/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ca5fed85b4fc975"}},{"code_sha256_prefix":"7e90a063f37f4fca","entry":"get_imagenet_model","repo":"renchunzi-xie/mano","repo_kind":"official","path":"models/utils.py","file_url":"https://github.com/renchunzi-xie/mano/blob/HEAD/models/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e90a063f37f4fca"}},{"code_sha256_prefix":"ff016b0b64047524","entry":"get_model","repo":"renchunzi-xie/mano","repo_kind":"official","path":"models/utils.py","file_url":"https://github.com/renchunzi-xie/mano/blob/HEAD/models/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ff016b0b64047524"}},{"code_sha256_prefix":"992a3c46f3a24a49","entry":"model_dataset_from_store","repo":"renchunzi-xie/mano","repo_kind":"official","path":"robustness1/model_utils.py","file_url":"https://github.com/renchunzi-xie/mano/blob/HEAD/robustness1/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"992a3c46f3a24a49"}},{"code_sha256_prefix":"5b04001dec8a2695","entry":"train","repo":"renchunzi-xie/mano","repo_kind":"official","path":"init_base_model.py","file_url":"https://github.com/renchunzi-xie/mano/blob/HEAD/init_base_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b04001dec8a2695"}},{"code_sha256_prefix":"d580012f4ab76cb9","entry":"wrn_50_2","repo":"renchunzi-xie/mano","repo_kind":"official","path":"models/wrn.py","file_url":"https://github.com/renchunzi-xie/mano/blob/HEAD/models/wrn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d580012f4ab76cb9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}