{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mmd-aggregated-two-sample-test","title":"MMD Aggregated Two-Sample Test","arxiv_id":"2110.15073","date":"2021-10-28","proceeding":"NeurIPS 2023 11","authors":["Antonin Schrab","Ilmun Kim","Mélisande Albert","Béatrice Laurent","Benjamin Guedj","Arthur Gretton"],"abstract":"We propose two novel nonparametric two-sample kernel tests based on the Maximum Mean Discrepancy (MMD). First, for a fixed kernel, we construct an MMD test using either permutations or a wild bootstrap, two popular numerical procedures to determine the test threshold. We prove that this test controls the probability of type I error non-asymptotically. Hence, it can be used reliably even in settings with small sample sizes as it remains well-calibrated, which differs from previous MMD tests which only guarantee correct test level asymptotically. When the difference in densities lies in a Sobolev ball, we prove minimax optimality of our MMD test with a specific kernel depending on the smoothness parameter of the Sobolev ball. In practice, this parameter is unknown and, hence, the optimal MMD test with this particular kernel cannot be used. To overcome this issue, we construct an aggregated test, called MMDAgg, which is adaptive to the smoothness parameter. The test power is maximised over the collection of kernels used, without requiring held-out data for kernel selection (which results in a loss of test power), or arbitrary kernel choices such as the median heuristic. We prove that MMDAgg still controls the level non-asymptotically, and achieves the minimax rate over Sobolev balls, up to an iterated logarithmic term. Our guarantees are not restricted to a specific type of kernel, but hold for any product of one-dimensional translation invariant characteristic kernels. We provide a user-friendly parameter-free implementation of MMDAgg using an adaptive collection of bandwidths. We demonstrate that MMDAgg significantly outperforms alternative state-of-the-art MMD-based two-sample tests on synthetic data satisfying the Sobolev smoothness assumption, and that, on real-world image data, MMDAgg closely matches the power of tests leveraging the use of models such as neural networks.","url_abs":"https://arxiv.org/abs/2110.15073v4","url_pdf":"https://arxiv.org/pdf/2110.15073v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mmd-aggregated-two-sample-test","repo_url":"https://github.com/antoninschrab/mmdagg","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"mmd-aggregated-two-sample-test","repo_url":"https://github.com/antoninschrab/mmdagg-paper","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"mmd-aggregated-two-sample-test","repo_url":"https://github.com/antoninschrab/fl-mmdagg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"translation","task_name":"Translation"},{"task_slug":"hypothesis-testing","task_name":"Two-sample testing"},{"task_slug":"two","task_name":"Vocal Bursts Valence Prediction"}],"methods":[{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.15073","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.15073"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/antoninschrab/mmdagg","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/antoninschrab/fl-mmdagg","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/antoninschrab/mmdagg-paper","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":3,"ran_honours":1,"unverified":11},"by_repo_kind":{"named_in_paper":{"samples":15,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"139c7796806ebcb2","entry":"compute_pairwise_matrix","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"mmdagg/np.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/mmdagg/np.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"139c7796806ebcb2"}},{"code_sha256_prefix":"ade4e10146ee1a5c","entry":"create_weights","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"mmdagg/jax.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/mmdagg/jax.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ade4e10146ee1a5c"}},{"code_sha256_prefix":"2c48c4eb0714eef3","entry":"kernel_matrix","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"mmdagg/jax.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/mmdagg/jax.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c48c4eb0714eef3"}},{"code_sha256_prefix":"4e7beb2199214856","entry":"kernel_matrix","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"mmdagg/np.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/mmdagg/np.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4e7beb2199214856"}},{"code_sha256_prefix":"74f5342c2c79173c","entry":"G","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"sampling.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/sampling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74f5342c2c79173c"}},{"code_sha256_prefix":"7b05376c6740b314","entry":"compute_median_bandwidth","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"median.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/median.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b05376c6740b314"}},{"code_sha256_prefix":"de5787afebcb8a7c","entry":"compute_median_bandwidth_subset","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"median.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/median.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de5787afebcb8a7c"}},{"code_sha256_prefix":"3de090d02c055924","entry":"f0","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"sampling.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/sampling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3de090d02c055924"}},{"code_sha256_prefix":"8f3b20cf3cb686c2","entry":"f_theta","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"sampling.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/sampling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f3b20cf3cb686c2"}},{"code_sha256_prefix":"43b365de0cc59f50","entry":"kernel_matrices","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"kernel.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/kernel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43b365de0cc59f50"}},{"code_sha256_prefix":"51da116869a49b91","entry":"mmdagg","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"mmdagg/jax.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/mmdagg/jax.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51da116869a49b91"}},{"code_sha256_prefix":"dac4409612e19fc2","entry":"pairwise_l1_distance","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"kernel.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/kernel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dac4409612e19fc2"}},{"code_sha256_prefix":"a73df06f2cbc35e5","entry":"pairwise_square_l2_distance","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"kernel.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/kernel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a73df06f2cbc35e5"}},{"code_sha256_prefix":"7da2a914d0ee3daf","entry":"truncated_gaussian","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"ost.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/ost.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7da2a914d0ee3daf"}},{"code_sha256_prefix":"b8733b899ab4f726","entry":"truncation","repo":"antoninschrab/mmdagg-paper","repo_kind":"named_in_paper","path":"ost.py","file_url":"https://github.com/antoninschrab/mmdagg-paper/blob/HEAD/ost.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b8733b899ab4f726"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}