{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/predictive-information-accelerates-learning","title":"Predictive Information Accelerates Learning in RL","arxiv_id":"2007.12401","date":"2020-07-24","proceeding":"NeurIPS 2020 12","authors":["Kuang-Huei Lee","Ian Fischer","Anthony Liu","Yijie Guo","Honglak Lee","John Canny","Sergio Guadarrama"],"abstract":"The Predictive Information is the mutual information between the past and the future, I(X_past; X_future). We hypothesize that capturing the predictive information is useful in RL, since the ability to model what will happen next is necessary for success on many tasks. To test our hypothesis, we train Soft Actor-Critic (SAC) agents from pixels with an auxiliary task that learns a compressed representation of the predictive information of the RL environment dynamics using a contrastive version of the Conditional Entropy Bottleneck (CEB) objective. We refer to these as Predictive Information SAC (PI-SAC) agents. We show that PI-SAC agents can substantially improve sample efficiency over challenging baselines on tasks from the DM Control suite of continuous control environments. We evaluate PI-SAC agents by comparing against uncompressed PI-SAC agents, other compressed and uncompressed agents, and SAC agents directly trained from pixels. Our implementation is given on GitHub.","url_abs":"https://arxiv.org/abs/2007.12401v2","url_pdf":"https://arxiv.org/pdf/2007.12401v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"predictive-information-accelerates-learning","repo_url":"https://github.com/google-research/pisac","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"continuous-control","task_name":"continuous-control"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dilated-convolution","method_name":"Dilated Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"sac","method_name":"SAC"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.12401","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.12401"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research/pisac","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"31b1ba8f3801d92a","entry":"expd","repo":"google-research/pisac","repo_kind":"official","path":"pisac/schedule_utils.py","file_url":"https://github.com/google-research/pisac/blob/HEAD/pisac/schedule_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"31b1ba8f3801d92a"}},{"code_sha256_prefix":"6084facad2835cd4","entry":"filter_invalid_transition","repo":"google-research/pisac","repo_kind":"official","path":"pisac/utils.py","file_url":"https://github.com/google-research/pisac/blob/HEAD/pisac/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6084facad2835cd4"}},{"code_sha256_prefix":"b8a75ea16d98165d","entry":"get_schedule_fn","repo":"google-research/pisac","repo_kind":"official","path":"pisac/schedule_utils.py","file_url":"https://github.com/google-research/pisac/blob/HEAD/pisac/schedule_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b8a75ea16d98165d"}},{"code_sha256_prefix":"01b60cf975a07343","entry":"image_strip_summary","repo":"google-research/pisac","repo_kind":"official","path":"pisac/utils.py","file_url":"https://github.com/google-research/pisac/blob/HEAD/pisac/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"01b60cf975a07343"}},{"code_sha256_prefix":"a42ce42da2f7b6c5","entry":"schedule_from_str","repo":"google-research/pisac","repo_kind":"official","path":"pisac/schedule_utils.py","file_url":"https://github.com/google-research/pisac/blob/HEAD/pisac/schedule_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a42ce42da2f7b6c5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}