{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/verifiable-evaluations-of-machine-learning","title":"Verifiable evaluations of machine learning models using zkSNARKs","arxiv_id":"2402.02675","date":"2024-02-05","proceeding":null,"authors":["Tobin South","Alexander Camuto","Shrey Jain","Shayla Nguyen","Robert Mahari","Christian Paquin","Jason Morton","Alex 'Sandy' Pentland"],"abstract":"In a world of increasing closed-source commercial machine learning models, model evaluations from developers must be taken at face value. These benchmark results-whether over task accuracy, bias evaluations, or safety checks-are traditionally impossible to verify by a model end-user without the costly or impossible process of re-performing the benchmark on black-box model outputs. This work presents a method of verifiable model evaluation using model inference through zkSNARKs. The resulting zero-knowledge computational proofs of model outputs over datasets can be packaged into verifiable evaluation attestations showing that models with fixed private weights achieve stated performance or fairness metrics over public inputs. We present a flexible proving system that enables verifiable attestations to be performed on any standard neural network model with varying compute requirements. For the first time, we demonstrate this across a sample of real-world models and highlight key challenges and design solutions. This presents a new transparency paradigm in the verifiable evaluation of private models.","url_abs":"https://arxiv.org/abs/2402.02675v2","url_pdf":"https://arxiv.org/pdf/2402.02675v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"verifiable-evaluations-of-machine-learning","repo_url":"https://github.com/tobinsouth/verifiableevals","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.02675","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.02675"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tobinsouth/verifiableevals","reach":null}],"summary":{"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9e1aae49417b8d42","entry":"calculate_differences","repo":"tobinsouth/verifiableevals","repo_kind":"official","path":"src/experiments/accuracy/accuracy_measurements.py","file_url":"https://github.com/tobinsouth/verifiableevals/blob/HEAD/src/experiments/accuracy/accuracy_measurements.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e1aae49417b8d42"}},{"code_sha256_prefix":"9a7d23f630ced1ca","entry":"input_file_to_io","repo":"tobinsouth/verifiableevals","repo_kind":"official","path":"src/experiments/accuracy/accuracy_measurements.py","file_url":"https://github.com/tobinsouth/verifiableevals/blob/HEAD/src/experiments/accuracy/accuracy_measurements.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a7d23f630ced1ca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}