{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reinforcement-learning-for-temporal-logic","title":"Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees","arxiv_id":"1909.05304","date":"2019-09-11","proceeding":null,"authors":["Mohammadhosein Hasanbeig","Yiannis Kantaros","Alessandro Abate","Daniel Kroening","George J. Pappas","Insup Lee"],"abstract":"Reinforcement Learning (RL) has emerged as an efficient method of choice for solving complex sequential decision making problems in automatic control, computer science, economics, and biology. In this paper we present a model-free RL algorithm to synthesize control policies that maximize the probability of satisfying high-level control objectives given as Linear Temporal Logic (LTL) formulas. Uncertainty is considered in the workspace properties, the structure of the workspace, and the agent actions, giving rise to a Probabilistically-Labeled Markov Decision Process (PL-MDP) with unknown graph structure and stochastic behaviour, which is even more general case than a fully unknown MDP. We first translate the LTL specification into a Limit Deterministic Buchi Automaton (LDBA), which is then used in an on-the-fly product with the PL-MDP. Thereafter, we define a synchronous reward function based on the acceptance condition of the LDBA. Finally, we show that the RL algorithm delivers a policy that maximizes the satisfaction probability asymptotically. We provide experimental results that showcase the efficiency of the proposed method.","url_abs":"https://arxiv.org/abs/1909.05304v1","url_pdf":"https://arxiv.org/pdf/1909.05304v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reinforcement-learning-for-temporal-logic","repo_url":"https://github.com/grockious/lcrl","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"decision-making-under-uncertainty","task_name":"Decision Making Under Uncertainty"},{"task_slug":"hierarchical-reinforcement-learning","task_name":"Hierarchical Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"safe-reinforcement-learning","task_name":"Safe Reinforcement Learning"},{"task_slug":"sequential-decision-making","task_name":"Sequential Decision Making"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1909.05304","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1909.05304"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/grockious/lcrl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"13cccf56f08b0d20","entry":"step","repo":"grockious/lcrl","repo_kind":"official","path":"src/lcrl/automata/cart_pole.py","file_url":"https://github.com/grockious/lcrl/blob/HEAD/src/lcrl/automata/cart_pole.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13cccf56f08b0d20"}},{"code_sha256_prefix":"c28303defaa68021","entry":"step","repo":"grockious/lcrl","repo_kind":"official","path":"src/lcrl/automata/frozen_lake_1_2_3.py","file_url":"https://github.com/grockious/lcrl/blob/HEAD/src/lcrl/automata/frozen_lake_1_2_3.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c28303defaa68021"}},{"code_sha256_prefix":"fe5a29213a5ed23d","entry":"step","repo":"grockious/lcrl","repo_kind":"official","path":"src/lcrl/automata/frozen_lake_4_5_6.py","file_url":"https://github.com/grockious/lcrl/blob/HEAD/src/lcrl/automata/frozen_lake_4_5_6.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe5a29213a5ed23d"}},{"code_sha256_prefix":"6615f68ab5ba1c9b","entry":"step","repo":"grockious/lcrl","repo_kind":"official","path":"src/lcrl/automata/goal1_or_goal2.py","file_url":"https://github.com/grockious/lcrl/blob/HEAD/src/lcrl/automata/goal1_or_goal2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6615f68ab5ba1c9b"}},{"code_sha256_prefix":"d51a7a42f1ec1a1b","entry":"step","repo":"grockious/lcrl","repo_kind":"official","path":"src/lcrl/automata/goal1_then_goal2.py","file_url":"https://github.com/grockious/lcrl/blob/HEAD/src/lcrl/automata/goal1_then_goal2.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d51a7a42f1ec1a1b"}},{"code_sha256_prefix":"c6ec4fbec67ad889","entry":"step","repo":"grockious/lcrl","repo_kind":"official","path":"src/lcrl/automata/mars_rover_1_3.py","file_url":"https://github.com/grockious/lcrl/blob/HEAD/src/lcrl/automata/mars_rover_1_3.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c6ec4fbec67ad889"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}