{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-agent-generalized-recursive-reasoning","title":"Modelling Bounded Rationality in Multi-Agent Interactions by Generalized Recursive Reasoning","arxiv_id":"1901.09216","date":"2019-01-26","proceeding":null,"authors":["Ying Wen","Yaodong Yang","Rui Luo","Jun Wang"],"abstract":"Though limited in real-world decision making, most multi-agent reinforcement learning (MARL) models assume perfectly rational agents -- a property hardly met due to individual's cognitive limitation and/or the tractability of the decision problem. In this paper, we introduce generalized recursive reasoning (GR2) as a novel framework to model agents with different \\emph{hierarchical} levels of rationality; our framework enables agents to exhibit varying levels of \"thinking\" ability thereby allowing higher-level agents to best respond to various less sophisticated learners. We contribute both theoretically and empirically. On the theory side, we devise the hierarchical framework of GR2 through probabilistic graphical models and prove the existence of a perfect Bayesian equilibrium. Within the GR2, we propose a practical actor-critic solver, and demonstrate its convergent property to a stationary point in two-player games through Lyapunov analysis. On the empirical side, we validate our findings on a variety of MARL benchmarks. Precisely, we first illustrate the hierarchical thinking process on the Keynes Beauty Contest, and then demonstrate significant improvements compared to state-of-the-art opponent modeling baselines on the normal-form games and the cooperative navigation benchmark.","url_abs":"https://arxiv.org/abs/1901.09216v2","url_pdf":"https://arxiv.org/pdf/1901.09216v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-agent-generalized-recursive-reasoning","repo_url":"https://github.com/ying-wen/gr2","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"multi-agent-reinforcement-learning","task_name":"Multi-agent Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1901.09216","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1901.09216"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ying-wen/gr2","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_honours":1,"unverified":8},"by_repo_kind":{"official":{"samples":10,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"aad5113d0b095522","entry":"shape_el","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/distributions/common.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/distributions/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aad5113d0b095522"}},{"code_sha256_prefix":"91fe7a23c1c052b7","entry":"softmax","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/utils.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"91fe7a23c1c052b7"}},{"code_sha256_prefix":"ba45f82a48ab3a78","entry":"annotate_heatmap","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/plot.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba45f82a48ab3a78"}},{"code_sha256_prefix":"e20241ffb27fdd20","entry":"checkerboard","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/distributions/real_nvp_bijector.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/distributions/real_nvp_bijector.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e20241ffb27fdd20"}},{"code_sha256_prefix":"2a88c74b929c3777","entry":"conditioned_real_nvp_template","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/distributions/real_nvp_flow.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/distributions/real_nvp_flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a88c74b929c3777"}},{"code_sha256_prefix":"026ed64900497cf5","entry":"deep_clone","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/core/serializable.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/core/serializable.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"026ed64900497cf5"}},{"code_sha256_prefix":"c72e67905cf81bd6","entry":"feedforward_net","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/distributions/real_nvp_bijector.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/distributions/real_nvp_bijector.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c72e67905cf81bd6"}},{"code_sha256_prefix":"3ef71138411e0095","entry":"heatmap","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/plot.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ef71138411e0095"}},{"code_sha256_prefix":"a292492441660b58","entry":"projection","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/utils.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a292492441660b58"}},{"code_sha256_prefix":"db4eab2810a17578","entry":"sigmoid","repo":"ying-wen/gr2","repo_kind":"official","path":"code/maci/utils.py","file_url":"https://github.com/ying-wen/gr2/blob/HEAD/code/maci/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db4eab2810a17578"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}