{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-incentivize-other-learning-agents","title":"Learning to Incentivize Other Learning Agents","arxiv_id":"2006.06051","date":"2020-06-10","proceeding":"NeurIPS 2020 12","authors":["Jiachen Yang","Ang Li","Mehrdad Farajtabar","Peter Sunehag","Edward Hughes","Hongyuan Zha"],"abstract":"The challenge of developing powerful and general Reinforcement Learning (RL) agents has received increasing attention in recent years. Much of this effort has focused on the single-agent setting, in which an agent maximizes a predefined extrinsic reward function. However, a long-term question inevitably arises: how will such independent agents cooperate when they are continually learning and acting in a shared multi-agent environment? Observing that humans often provide incentives to influence others' behavior, we propose to equip each RL agent in a multi-agent environment with the ability to give rewards directly to other agents, using a learned incentive function. Each agent learns its own incentive function by explicitly accounting for its impact on the learning of recipients and, through them, the impact on its own extrinsic objective. We demonstrate in experiments that such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games, often by finding a near-optimal division of labor. Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future.","url_abs":"https://arxiv.org/abs/2006.06051v2","url_pdf":"https://arxiv.org/pdf/2006.06051v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-incentivize-other-learning-agents","repo_url":"https://github.com/011235813/lio","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"learning-to-incentivize-other-learning-agents","repo_url":"https://github.com/timmatheis/learning-to-participate","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"general-reinforcement-learning","task_name":"General Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.06051","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.06051"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/timmatheis/learning-to-participate","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/011235813/lio","reach":null}],"summary":{"unverified":10},"by_repo_kind":{"official":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c247fc92e96cd594","entry":"LIO","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c247fc92e96cd594"}},{"code_sha256_prefix":"574215b6bf00a5c6","entry":"PolicyNewCNN","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"574215b6bf00a5c6"}},{"code_sha256_prefix":"151d6b6764b8b9a5","entry":"actor","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"151d6b6764b8b9a5"}},{"code_sha256_prefix":"436aeee22a5c95dd","entry":"actor_mlp","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"436aeee22a5c95dd"}},{"code_sha256_prefix":"135cb40c9af85d4b","entry":"conv","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"135cb40c9af85d4b"}},{"code_sha256_prefix":"cdfe44b4e9b56c5a","entry":"convnet","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cdfe44b4e9b56c5a"}},{"code_sha256_prefix":"fb4097e566da6fc0","entry":"reward","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb4097e566da6fc0"}},{"code_sha256_prefix":"85fe60ad96240c2d","entry":"reward_mlp","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"85fe60ad96240c2d"}},{"code_sha256_prefix":"7f8cd187adfa8a9b","entry":"vnet","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f8cd187adfa8a9b"}},{"code_sha256_prefix":"43b688c27d0fd391","entry":"vnet_mlp","repo":"011235813/lio","repo_kind":"official","path":"lio/alg/lio_ac.py","file_url":"https://github.com/011235813/lio/blob/HEAD/lio/alg/lio_ac.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43b688c27d0fd391"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}