{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scalable-bayesian-inverse-reinforcement-1","title":"Scalable Bayesian Inverse Reinforcement Learning","arxiv_id":"2102.06483","date":"2021-02-12","proceeding":null,"authors":["Alex J. Chan","Mihaela van der Schaar"],"abstract":"Bayesian inference over the reward presents an ideal solution to the ill-posed nature of the inverse reinforcement learning problem. Unfortunately current methods generally do not scale well beyond the small tabular setting due to the need for an inner-loop MDP solver, and even non-Bayesian methods that do themselves scale often require extensive interaction with the environment to perform well, being inappropriate for high stakes or costly applications such as healthcare. In this paper we introduce our method, Approximate Variational Reward Imitation Learning (AVRIL), that addresses both of these issues by jointly learning an approximate posterior distribution over the reward that scales to arbitrarily complicated state spaces alongside an appropriate policy in a completely offline manner through a variational approach to said latent reward. Applying our method to real medical data alongside classic control simulations, we demonstrate Bayesian reward inference in environments beyond the scope of current methods, as well as task performance competitive with focused offline imitation learning algorithms.","url_abs":"https://arxiv.org/abs/2102.06483v2","url_pdf":"https://arxiv.org/pdf/2102.06483v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scalable-bayesian-inverse-reinforcement-1","repo_url":"https://github.com/XanderJC/scalable-birl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":null},{"paper_slug":"scalable-bayesian-inverse-reinforcement-1","repo_url":"https://github.com/vanderschaarlab/mlforhealthlabpub","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"imitation-learning","task_name":"Imitation Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.06483","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2102.06483"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/XanderJC/scalable-birl","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vanderschaarlab/mlforhealthlabpub","reach":null}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d01966723d682fa9","entry":"encoder_model","repo":"XanderJC/scalable-birl","repo_kind":"official","path":"sbirl/models.py","file_url":"https://github.com/XanderJC/scalable-birl/blob/HEAD/sbirl/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d01966723d682fa9"}},{"code_sha256_prefix":"967dc395467302de","entry":"hidden_layers","repo":"XanderJC/scalable-birl","repo_kind":"official","path":"sbirl/models.py","file_url":"https://github.com/XanderJC/scalable-birl/blob/HEAD/sbirl/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"967dc395467302de"}},{"code_sha256_prefix":"a3e6718af33545aa","entry":"load_data","repo":"XanderJC/scalable-birl","repo_kind":"official","path":"sbirl/utils.py","file_url":"https://github.com/XanderJC/scalable-birl/blob/HEAD/sbirl/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3e6718af33545aa"}},{"code_sha256_prefix":"0def68b781b37765","entry":"q_network_model","repo":"XanderJC/scalable-birl","repo_kind":"official","path":"sbirl/models.py","file_url":"https://github.com/XanderJC/scalable-birl/blob/HEAD/sbirl/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0def68b781b37765"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}