{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/timetraveler-reinforcement-learning-for","title":"TimeTraveler: Reinforcement Learning for Temporal Knowledge Graph Forecasting","arxiv_id":"2109.04101","date":"2021-09-09","proceeding":"EMNLP 2021 11","authors":["Haohai Sun","Jialun Zhong","Yunpu Ma","Zhen Han","Kun He"],"abstract":"Temporal knowledge graph (TKG) reasoning is a crucial task that has gained increasing research interest in recent years. Most existing methods focus on reasoning at past timestamps to complete the missing facts, and there are only a few works of reasoning on known TKGs to forecast future facts. Compared with the completion task, the forecasting task is more difficult that faces two main challenges: (1) how to effectively model the time information to handle future timestamps? (2) how to make inductive inference to handle previously unseen entities that emerge over time? To address these challenges, we propose the first reinforcement learning method for forecasting. Specifically, the agent travels on historical knowledge graph snapshots to search for the answer. Our method defines a relative time encoding function to capture the timespan information, and we design a novel time-shaped reward based on Dirichlet distribution to guide the model learning. Furthermore, we propose a novel representation method for unseen entities to improve the inductive inference ability of the model. We evaluate our method for this link prediction task at future timestamps. Extensive experiments on four benchmark datasets demonstrate substantial performance improvement meanwhile with higher explainability, less calculation, and fewer parameters when compared with existing state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2109.04101v1","url_pdf":"https://arxiv.org/pdf/2109.04101v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"timetraveler-reinforcement-learning-for","repo_url":"https://github.com/jhl-hust/titer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"link-prediction","task_name":"Link Prediction"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2109.04101","atlas_url":"https://app.syntology.ai/?focus=2109.04101","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2109.04101"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jhl-hust/titer","reach":null}],"summary":{"ran":3,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"b78447b20f062c60","entry":"DynamicEmbedding","repo":"jhl-hust/titer","repo_kind":"official","path":"model/agent.py","file_url":"https://github.com/jhl-hust/titer/blob/HEAD/model/agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b78447b20f062c60"}},{"code_sha256_prefix":"7865b8741ed3470f","entry":"HistoryEncoder","repo":"jhl-hust/titer","repo_kind":"official","path":"model/agent.py","file_url":"https://github.com/jhl-hust/titer/blob/HEAD/model/agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7865b8741ed3470f"}},{"code_sha256_prefix":"5a8cc407b3dae52f","entry":"PolicyMLP","repo":"jhl-hust/titer","repo_kind":"official","path":"model/agent.py","file_url":"https://github.com/jhl-hust/titer/blob/HEAD/model/agent.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5a8cc407b3dae52f"}},{"code_sha256_prefix":"25b071340b057db5","entry":"Agent","repo":"jhl-hust/titer","repo_kind":"official","path":"model/agent.py","file_url":"https://github.com/jhl-hust/titer/blob/HEAD/model/agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"25b071340b057db5"}},{"code_sha256_prefix":"807e28c4b2595486","entry":"StaticEmbedding","repo":"jhl-hust/titer","repo_kind":"official","path":"model/agent.py","file_url":"https://github.com/jhl-hust/titer/blob/HEAD/model/agent.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"807e28c4b2595486"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}