{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-test-of-relative-similarity-for-model","title":"A Test of Relative Similarity For Model Selection in Generative Models","arxiv_id":"1511.04581","date":"2015-11-14","proceeding":null,"authors":["Wacha Bounliphone","Eugene Belilovsky","Matthew B. Blaschko","Ioannis Antonoglou","Arthur Gretton"],"abstract":"Probabilistic generative models provide a powerful framework for representing\ndata that avoids the expense of manual annotation typically needed by\ndiscriminative approaches. Model selection in this generative setting can be\nchallenging, however, particularly when likelihoods are not easily accessible.\nTo address this issue, we introduce a statistical test of relative similarity,\nwhich is used to determine which of two models generates samples that are\nsignificantly closer to a real-world reference dataset of interest. We use as\nour test statistic the difference in maximum mean discrepancies (MMDs) between\nthe reference dataset and each model dataset, and derive a powerful,\nlow-variance test based on the joint asymptotic distribution of the MMDs\nbetween each reference-model pair. In experiments on deep generative models,\nincluding the variational auto-encoder and generative moment matching network,\nthe tests provide a meaningful ranking of model performance as a function of\nparameter and training settings.","url_abs":"http://arxiv.org/abs/1511.04581v4","url_pdf":"http://arxiv.org/pdf/1511.04581v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-test-of-relative-similarity-for-model","repo_url":"https://github.com/eugenium/MMD","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"model-selection","task_name":"Model Selection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1511.04581","atlas_url":"https://app.syntology.ai/?focus=1511.04581","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1511.04581"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eugenium/MMD","reach":null}],"summary":{"ran_honours":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"36f96addab29ba19","entry":"grbf","repo":"eugenium/MMD","repo_kind":"official","path":"mmd.py","file_url":"https://github.com/eugenium/MMD/blob/HEAD/mmd.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36f96addab29ba19"}},{"code_sha256_prefix":"3cc92f05cbac7f13","entry":"MMD_3_Sample_Test","repo":"eugenium/MMD","repo_kind":"official","path":"mmd.py","file_url":"https://github.com/eugenium/MMD/blob/HEAD/mmd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3cc92f05cbac7f13"}},{"code_sha256_prefix":"4fa0660abad83e3e","entry":"MMD_Diff_Var","repo":"eugenium/MMD","repo_kind":"official","path":"mmd.py","file_url":"https://github.com/eugenium/MMD/blob/HEAD/mmd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4fa0660abad83e3e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}