{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sampling-in-constrained-domains-with","title":"Sampling in Constrained Domains with Orthogonal-Space Variational Gradient Descent","arxiv_id":"2210.06447","date":"2022-10-12","proceeding":null,"authors":["Ruqi Zhang","Qiang Liu","Xin T. Tong"],"abstract":"Sampling methods, as important inference and learning techniques, are typically designed for unconstrained domains. However, constraints are ubiquitous in machine learning problems, such as those on safety, fairness, robustness, and many other properties that must be satisfied to apply sampling results in real-life applications. Enforcing these constraints often leads to implicitly-defined manifolds, making efficient sampling with constraints very challenging. In this paper, we propose a new variational framework with a designed orthogonal-space gradient flow (O-Gradient) for sampling on a manifold $\\mathcal{G}_0$ defined by general equality constraints. O-Gradient decomposes the gradient into two parts: one decreases the distance to $\\mathcal{G}_0$ and the other decreases the KL divergence in the orthogonal space. While most existing manifold sampling methods require initialization on $\\mathcal{G}_0$, O-Gradient does not require such prior knowledge. We prove that O-Gradient converges to the target constrained distribution with rate $\\widetilde{O}(1/\\text{the number of iterations})$ under mild conditions. Our proof relies on a new Stein characterization of conditional measure which could be of independent interest. We implement O-Gradient through both Langevin dynamics and Stein variational gradient descent and demonstrate its effectiveness in various experiments, including Bayesian deep neural networks.","url_abs":"https://arxiv.org/abs/2210.06447v1","url_pdf":"https://arxiv.org/pdf/2210.06447v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sampling-in-constrained-domains-with","repo_url":"https://github.com/ruqizhang/o-gradient","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.06447","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.06447"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ruqizhang/o-gradient","reach":null}],"summary":{"ran":1,"ran_honours":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"73f9844f4eb042cc","entry":"O_Langevin","repo":"ruqizhang/o-gradient","repo_kind":"official","path":"samplers.py","file_url":"https://github.com/ruqizhang/o-gradient/blob/HEAD/samplers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"73f9844f4eb042cc"}},{"code_sha256_prefix":"3b15204af8874b1a","entry":"compute_w2","repo":"ruqizhang/o-gradient","repo_kind":"official","path":"main_synthetic.py","file_url":"https://github.com/ruqizhang/o-gradient/blob/HEAD/main_synthetic.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3b15204af8874b1a"}},{"code_sha256_prefix":"19a0a20172bf427d","entry":"plot_2d_zero","repo":"ruqizhang/o-gradient","repo_kind":"official","path":"main_synthetic.py","file_url":"https://github.com/ruqizhang/o-gradient/blob/HEAD/main_synthetic.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19a0a20172bf427d"}},{"code_sha256_prefix":"31a4a526653a1689","entry":"adjust_learning_rate","repo":"ruqizhang/o-gradient","repo_kind":"official","path":"main_bnn_langevin.py","file_url":"https://github.com/ruqizhang/o-gradient/blob/HEAD/main_bnn_langevin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"31a4a526653a1689"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}