{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/blending-controllers-via-multi-objective","title":"Blending Controllers via Multi-Objective Bandits","arxiv_id":"2007.15755","date":"2020-07-30","proceeding":null,"authors":["Parham Gohari","Franck Djeumou","Abraham P. Vinod","Ufuk Topcu"],"abstract":"Safety and performance are often two competing objectives in sequential decision-making problems. Existing performant controllers, such as controllers derived from reinforcement learning algorithms, often fall short of safety guarantees. On the contrary, controllers that guarantee safety, such as those derived from classical control theory, require restrictive assumptions and are often conservative in performance. Our goal is to blend a performant and a safe controller to generate a single controller that is safer than the performant and accumulates higher rewards than the safe controller. To this end, we propose a blending algorithm using the framework of contextual multi-armed multi-objective bandits. At each stage, the algorithm observes the environment's current context alongside an immediate reward and cost, which is the underlying safety measure. The algorithm then decides which controller to employ based on its observations. We demonstrate that the algorithm achieves sublinear Pareto regret, a performance measure that models coherence with an expert that always avoids picking the controller with both inferior safety and performance. We derive an upper bound on the loss in individual objectives, which imposes no additional computational complexity. We empirically demonstrate the algorithm's success in blending a safe and a performant controller in a safety-focused testbed, the Safety Gym environment. A statistical analysis of the blended controller's total reward and cost reflects two key takeaways: The blended controller shows a strict improvement in performance compared to the safe controller, and it is safer than the performant controller.","url_abs":"http://arxiv.org/abs/2007.15755v1","url_pdf":"http://arxiv.org/pdf/2007.15755v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"blending-controllers-via-multi-objective","repo_url":"https://github.com/parhamgohari/blending-controllers","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.15755","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.15755"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/parhamgohari/blending-controllers","reach":null}],"summary":{"ran_fixture":2,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"f1cd4347111a0a00","entry":"compute_pareto_nd","repo":"parhamgohari/blending-controllers","repo_kind":"official","path":"blending_algo.py","file_url":"https://github.com/parhamgohari/blending-controllers/blob/HEAD/blending_algo.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f1cd4347111a0a00"}},{"code_sha256_prefix":"fa73c12882481f7d","entry":"feat_map","repo":"parhamgohari/blending-controllers","repo_kind":"official","path":"blending_algo.py","file_url":"https://github.com/parhamgohari/blending-controllers/blob/HEAD/blending_algo.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa73c12882481f7d"}},{"code_sha256_prefix":"45931986e0c5278f","entry":"feat_map_nstep","repo":"parhamgohari/blending-controllers","repo_kind":"official","path":"blending_algo.py","file_url":"https://github.com/parhamgohari/blending-controllers/blob/HEAD/blending_algo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45931986e0c5278f"}},{"code_sha256_prefix":"31a1b7aeee6efe99","entry":"compute_pgap","repo":"parhamgohari/blending-controllers","repo_kind":"official","path":"blending_algo.py","file_url":"https://github.com/parhamgohari/blending-controllers/blob/HEAD/blending_algo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"31a1b7aeee6efe99"}},{"code_sha256_prefix":"34e68423bc34309a","entry":"one_trace_blending_nstep","repo":"parhamgohari/blending-controllers","repo_kind":"official","path":"blending_algo.py","file_url":"https://github.com/parhamgohari/blending-controllers/blob/HEAD/blending_algo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34e68423bc34309a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}