{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/counterfactual-control-for-free-from","title":"Counterfactual Control for Free from Generative Models","arxiv_id":"1702.06676","date":"2017-02-22","proceeding":null,"authors":["Nicholas Guttenberg","Yen Yu","Ryota Kanai"],"abstract":"We introduce a method by which a generative model learning the joint\ndistribution between actions and future states can be used to automatically\ninfer a control scheme for any desired reward function, which may be altered on\nthe fly without retraining the model. In this method, the problem of action\nselection is reduced to one of gradient descent on the latent space of the\ngenerative model, with the model itself providing the means of evaluating\noutcomes and finding the gradient, much like how the reward network in Deep\nQ-Networks (DQN) provides gradient information for the action generator. Unlike\nDQN or Actor-Critic, which are conditional models for a specific reward, using\na generative model of the full joint distribution permits the reward to be\nchanged on the fly. In addition, the generated futures can be inspected to gain\ninsight in to what the network 'thinks' will happen, and to what went wrong\nwhen the outcomes deviate from prediction.","url_abs":"http://arxiv.org/abs/1702.06676v2","url_pdf":"http://arxiv.org/pdf/1702.06676v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"counterfactual-control-for-free-from","repo_url":"https://github.com/arayabrain/GenerativeControl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"counterfactual-control-for-free-from","repo_url":"https://github.com/BerenMillidge/GenerativeControlCartpole.jl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":null,"task_name":"counterfactual"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}