{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/interactively-learning-preference-constraints","title":"Interactively Learning Preference Constraints in Linear Bandits","arxiv_id":"2206.05255","date":"2022-06-10","proceeding":null,"authors":["David Lindner","Sebastian Tschiatschek","Katja Hofmann","Andreas Krause"],"abstract":"We study sequential decision-making with known rewards and unknown constraints, motivated by situations where the constraints represent expensive-to-evaluate human preferences, such as safe and comfortable driving behavior. We formalize the challenge of interactively learning about these constraints as a novel linear bandit problem which we call constrained linear best-arm identification. To solve this problem, we propose the Adaptive Constraint Learning (ACOL) algorithm. We provide an instance-dependent lower bound for constrained linear best-arm identification and show that ACOL's sample complexity matches the lower bound in the worst-case. In the average case, ACOL's sample complexity bound is still significantly tighter than bounds of simpler approaches. In synthetic experiments, ACOL performs on par with an oracle solution and outperforms a range of baselines. As an application, we consider learning constraints to represent human preferences in a driving simulation. ACOL is significantly more sample efficient than alternatives for this application. Further, we find that learning preferences as constraints is more robust to changes in the driving scenario than encoding the preferences directly in the reward function.","url_abs":"https://arxiv.org/abs/2206.05255v1","url_pdf":"https://arxiv.org/pdf/2206.05255v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"interactively-learning-preference-constraints","repo_url":"https://github.com/lasgroup/adaptive-constraint-learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"sequential-decision-making","task_name":"Sequential Decision Making"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.05255","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.05255"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/lasgroup/adaptive-constraint-learning","reach":null}],"summary":{"ran":1,"ran_violates":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8b23d2f771d8f5ac","entry":"CLBAdaptiveGreedyAlgorithm","repo":"lasgroup/adaptive-constraint-learning","repo_kind":"official","path":"src/constrained_bai/algorithms/adaptive_greedy.py","file_url":"https://github.com/lasgroup/adaptive-constraint-learning/blob/HEAD/src/constrained_bai/algorithms/adaptive_greedy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b23d2f771d8f5ac"}},{"code_sha256_prefix":"497a1f936b172919","entry":"compute_xnorm","repo":"lasgroup/adaptive-constraint-learning","repo_kind":"official","path":"src/constrained_bai/algorithms/adaptive_greedy.py","file_url":"https://github.com/lasgroup/adaptive-constraint-learning/blob/HEAD/src/constrained_bai/algorithms/adaptive_greedy.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"497a1f936b172919"}},{"code_sha256_prefix":"e8de59c1a0d9d1ce","entry":"CLBAlgorithm","repo":"lasgroup/adaptive-constraint-learning","repo_kind":"official","path":"src/constrained_bai/algorithms/adaptive_greedy.py","file_url":"https://github.com/lasgroup/adaptive-constraint-learning/blob/HEAD/src/constrained_bai/algorithms/adaptive_greedy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8de59c1a0d9d1ce"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}