{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/autotom-automated-bayesian-inverse-planning","title":"AutoToM: Automated Bayesian Inverse Planning and Model Discovery for Open-ended Theory of Mind","arxiv_id":"2502.15676","date":"2025-02-21","proceeding":null,"authors":["Zhining Zhang","Chuanyang Jin","Mung Yao Jia","Tianmin Shu"],"abstract":"Theory of Mind (ToM), the ability to understand people's mental variables based on their behavior, is key to developing socially intelligent agents. Current approaches to Theory of Mind reasoning either rely on prompting Large Language Models (LLMs), which are prone to systematic errors, or use rigid, handcrafted Bayesian Theory of Mind (BToM) models, which are more robust but cannot generalize across different domains. In this work, we introduce AutoToM, an automated Bayesian Theory of Mind method for achieving open-ended machine Theory of Mind. AutoToM can operate in any domain, infer any mental variable, and conduct robust Theory of Mind reasoning of any order. Given a Theory of Mind inference problem, AutoToM first proposes an initial BToM model. It then conducts automated Bayesian inverse planning based on the proposed model, leveraging an LLM as the backend. Based on the uncertainty of the inference, it iteratively refines the model, by introducing additional mental variables and/or incorporating more timesteps in the context. Empirical evaluations across multiple Theory of Mind benchmarks demonstrate that AutoToM consistently achieves state-of-the-art performance, offering a scalable, robust, and interpretable approach to machine Theory of Mind.","url_abs":"https://arxiv.org/abs/2502.15676v1","url_pdf":"https://arxiv.org/pdf/2502.15676v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"autotom-automated-bayesian-inverse-planning","repo_url":"https://github.com/SCAI-JHU/AutoToM","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"autotom-automated-bayesian-inverse-planning","repo_url":"https://github.com/chuanyangjin/MMToM-QA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"model-discovery","task_name":"Model Discovery"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.15676","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.15676"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chuanyangjin/MMToM-QA","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SCAI-JHU/AutoToM","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":2,"unverified":17},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1},"listed":{"samples":13,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c281d90797146140","entry":"capping_value","repo":"SCAI-JHU/AutoToM","repo_kind":"official","path":"model/BayesianInference.py","file_url":"https://github.com/SCAI-JHU/AutoToM/blob/HEAD/model/BayesianInference.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c281d90797146140"}},{"code_sha256_prefix":"e6a4ae1c85273c05","entry":"get_full_name","repo":"SCAI-JHU/AutoToM","repo_kind":"official","path":"experiment_2/food_truck_scenarios/eval_AutoToM.py","file_url":"https://github.com/SCAI-JHU/AutoToM/blob/HEAD/experiment_2/food_truck_scenarios/eval_AutoToM.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e6a4ae1c85273c05"}},{"code_sha256_prefix":"b59f27e924706f70","entry":"translate_truck","repo":"SCAI-JHU/AutoToM","repo_kind":"official","path":"experiment_2/food_truck_scenarios/eval_AutoToM.py","file_url":"https://github.com/SCAI-JHU/AutoToM/blob/HEAD/experiment_2/food_truck_scenarios/eval_AutoToM.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b59f27e924706f70"}},{"code_sha256_prefix":"4fae6c3fd70afc8a","entry":"BayesianInferenceModel","repo":"SCAI-JHU/AutoToM","repo_kind":"official","path":"model/BayesianInference.py","file_url":"https://github.com/SCAI-JHU/AutoToM/blob/HEAD/model/BayesianInference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4fae6c3fd70afc8a"}},{"code_sha256_prefix":"9d6967fc6d6fb660","entry":"filter_graph","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/graph_utils.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/graph_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d6967fc6d6fb660"}},{"code_sha256_prefix":"9e0462c3e4339871","entry":"generate_chat_response","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/testing_gpt.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/testing_gpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9e0462c3e4339871"}},{"code_sha256_prefix":"2f2910e1108a303e","entry":"generate_response","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/testing_gpt.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/testing_gpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f2910e1108a303e"}},{"code_sha256_prefix":"7a2201f8d94f0683","entry":"generate_response_gpt4v","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/testing_gpt.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/testing_gpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7a2201f8d94f0683"}},{"code_sha256_prefix":"9c51b4f6a608944e","entry":"get_id2node","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/graph_utils.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/graph_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c51b4f6a608944e"}},{"code_sha256_prefix":"193733e23f574eec","entry":"get_likelihood","repo":"SCAI-JHU/AutoToM","repo_kind":"official","path":"model/BayesianInference.py","file_url":"https://github.com/SCAI-JHU/AutoToM/blob/HEAD/model/BayesianInference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"193733e23f574eec"}},{"code_sha256_prefix":"fe1125ca1e0d2aff","entry":"get_likelihood_general","repo":"SCAI-JHU/AutoToM","repo_kind":"official","path":"model/BayesianInference.py","file_url":"https://github.com/SCAI-JHU/AutoToM/blob/HEAD/model/BayesianInference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe1125ca1e0d2aff"}},{"code_sha256_prefix":"91dd2e4b634cc8a5","entry":"load_pickles","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/generate_finetuning_dataset.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/generate_finetuning_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"91dd2e4b634cc8a5"}},{"code_sha256_prefix":"9fc6add2665e10d7","entry":"load_pickles","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/visual_perception.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/visual_perception.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9fc6add2665e10d7"}},{"code_sha256_prefix":"15217c48be12b1ec","entry":"main","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/inverse_symbolic_planner.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/inverse_symbolic_planner.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"15217c48be12b1ec"}},{"code_sha256_prefix":"8e2c4cea7bf40527","entry":"read_frame_intervals","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/visual_perception.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/visual_perception.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8e2c4cea7bf40527"}},{"code_sha256_prefix":"2d68812439daf499","entry":"remove_item","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/inverse_symbolic_planner.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/inverse_symbolic_planner.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d68812439daf499"}},{"code_sha256_prefix":"472ae09c86ea5f9a","entry":"see_truck","repo":"SCAI-JHU/AutoToM","repo_kind":"official","path":"experiment_2/food_truck_scenarios/eval_AutoToM.py","file_url":"https://github.com/SCAI-JHU/AutoToM/blob/HEAD/experiment_2/food_truck_scenarios/eval_AutoToM.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"OPAQUE_PARAMS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"472ae09c86ea5f9a"}},{"code_sha256_prefix":"145dfd845cadb346","entry":"split_actions","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/generate_finetuning_dataset.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/generate_finetuning_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"145dfd845cadb346"}},{"code_sha256_prefix":"e284acf374ca4072","entry":"transform_actions","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/generate_finetuning_dataset.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/generate_finetuning_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e284acf374ca4072"}},{"code_sha256_prefix":"305fbcb2b3de72c1","entry":"transform_graph","repo":"chuanyangjin/MMToM-QA","repo_kind":"listed","path":"BIP-ALM/graph_utils.py","file_url":"https://github.com/chuanyangjin/MMToM-QA/blob/HEAD/BIP-ALM/graph_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"305fbcb2b3de72c1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}