{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/kernel-semi-implicit-variational-inference","title":"Kernel Semi-Implicit Variational Inference","arxiv_id":"2405.18997","date":"2024-05-29","proceeding":null,"authors":["Ziheng Cheng","Longlin Yu","Tianyu Xie","Shiyue Zhang","Cheng Zhang"],"abstract":"Semi-implicit variational inference (SIVI) extends traditional variational families with semi-implicit distributions defined in a hierarchical manner. Due to the intractable densities of semi-implicit distributions, classical SIVI often resorts to surrogates of evidence lower bound (ELBO) that would introduce biases for training. A recent advancement in SIVI, named SIVI-SM, utilizes an alternative score matching objective made tractable via a minimax formulation, albeit requiring an additional lower-level optimization. In this paper, we propose kernel SIVI (KSIVI), a variant of SIVI-SM that eliminates the need for lower-level optimization through kernel tricks. Specifically, we show that when optimizing over a reproducing kernel Hilbert space (RKHS), the lower-level problem has an explicit solution. This way, the upper-level objective becomes the kernel Stein discrepancy (KSD), which is readily computable for stochastic gradient descent due to the hierarchical structure of semi-implicit variational distributions. An upper bound for the variance of the Monte Carlo gradient estimators of the KSD objective is derived, which allows us to establish novel convergence guarantees of KSIVI. We demonstrate the effectiveness and efficiency of KSIVI on both synthetic distributions and a variety of real data Bayesian inference tasks.","url_abs":"https://arxiv.org/abs/2405.18997v1","url_pdf":"https://arxiv.org/pdf/2405.18997v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"kernel-semi-implicit-variational-inference","repo_url":"https://github.com/longinyu/hsivi","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"kernel-semi-implicit-variational-inference","repo_url":"https://github.com/longinyu/ksivi","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.18997","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.18997"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/longinYu/KSIVI","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/longinyu/ksivi","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/longinyu/hsivi","reach":null}],"summary":{"ran":8,"ran_fixture":3,"ran_honours":1,"ran_draft_wrong":2,"unverified":2},"by_repo_kind":{"listed":{"samples":16,"ran":14,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"5601bc78a5228484","entry":"Banana_shape","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5601bc78a5228484"}},{"code_sha256_prefix":"482c1ee5ddf93185","entry":"Bnn","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"482c1ee5ddf93185"}},{"code_sha256_prefix":"5f7c89e1686bca8a","entry":"IMQ_kernel","repo":"longinyu/ksivi","repo_kind":"listed","path":"utils/kernels.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/utils/kernels.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f7c89e1686bca8a"}},{"code_sha256_prefix":"6542c19a838fa61d","entry":"LRwaveform","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6542c19a838fa61d"}},{"code_sha256_prefix":"991c97cb3b27313e","entry":"Langevin_post","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"991c97cb3b27313e"}},{"code_sha256_prefix":"57e3823e31fa77a9","entry":"Multimodal","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"57e3823e31fa77a9"}},{"code_sha256_prefix":"ee48748220daa480","entry":"SIMINet","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ee48748220daa480"}},{"code_sha256_prefix":"b7625bd511229985","entry":"StudentTFullDim","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b7625bd511229985"}},{"code_sha256_prefix":"eeb63522a2e3594d","entry":"X_shaped","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eeb63522a2e3594d"}},{"code_sha256_prefix":"eda37aa649362a3f","entry":"annealing","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eda37aa649362a3f"}},{"code_sha256_prefix":"946a6f88c2d5c17d","entry":"compute_jacobian","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"946a6f88c2d5c17d"}},{"code_sha256_prefix":"1f9e355031d1ccd1","entry":"density_estimation","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1f9e355031d1ccd1"}},{"code_sha256_prefix":"447cd6e519ab757f","entry":"gaussian_kernel","repo":"longinyu/ksivi","repo_kind":"listed","path":"utils/kernels.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/utils/kernels.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"447cd6e519ab757f"}},{"code_sha256_prefix":"d248c2131529471a","entry":"laplace_kernel","repo":"longinyu/ksivi","repo_kind":"listed","path":"utils/kernels.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/utils/kernels.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d248c2131529471a"}},{"code_sha256_prefix":"950572fc3dc84ed9","entry":"KernelSIVI_lr","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"950572fc3dc84ed9"}},{"code_sha256_prefix":"0ddcf131ce089894","entry":"Toy_2D","repo":"longinyu/ksivi","repo_kind":"listed","path":"sivistein_lr.py","file_url":"https://github.com/longinyu/ksivi/blob/HEAD/sivistein_lr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0ddcf131ce089894"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}