{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bark-open-behavior-benchmarking-in-multi","title":"BARK: Open Behavior Benchmarking in Multi-Agent Environments","arxiv_id":"2003.02604","date":"2020-03-05","proceeding":null,"authors":["Julian Bernhard","Klemens Esterle","Patrick Hart","Tobias Kessler"],"abstract":"Predicting and planning interactive behaviors in complex traffic situations presents a challenging task. Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own mission goal. As driving tests are costly and challenging scenarios are hard to find and reproduce, simulation is widely used to develop, test, and benchmark behavior models. However, most simulations rely on datasets and simplistic behavior models for traffic participants and do not cover the full variety of real-world, interactive human behaviors. In this work, we introduce BARK, an open-source behavior benchmarking environment designed to mitigate the shortcomings stated above. In BARK, behavior models are (re-)used for planning, prediction, and simulation. A range of models is currently available, such as Monte-Carlo Tree Search and Reinforcement Learning-based behavior models. We use a public dataset and sampling-based scenario generation to show the inter-exchangeability of behavior models in BARK. We evaluate how well the models used cope with interactions and how robust they are towards exchanging behavior models. Our evaluation shows that BARK provides a suitable framework for a systematic development of behavior models.","url_abs":"https://arxiv.org/abs/2003.02604v1","url_pdf":"https://arxiv.org/pdf/2003.02604v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"bark-open-behavior-benchmarking-in-multi","repo_url":"https://github.com/bark-simulator/bark","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"bark-open-behavior-benchmarking-in-multi","repo_url":"https://github.com/bark-simulator/bark-ml","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"bark-open-behavior-benchmarking-in-multi","repo_url":"https://github.com/bark-simulator/example_benchmark","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"bark-open-behavior-benchmarking-in-multi","repo_url":"https://github.com/2024-MindSpore-1/Code2/tree/main/model-1/bark","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2003.02604","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.02604"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/2024-MindSpore-1/Code2/tree/main/model-1/bark","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bark-simulator/example_benchmark","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bark-simulator/bark","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bark-simulator/bark-ml","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9eb70f020b27eabe","entry":"deserialize_benchmark_config","repo":"bark-simulator/bark","repo_kind":"official","path":"bark/benchmark/benchmark_runner_mp.py","file_url":"https://github.com/bark-simulator/bark/blob/HEAD/bark/benchmark/benchmark_runner_mp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9eb70f020b27eabe"}},{"code_sha256_prefix":"2cbc7b7a886e6b50","entry":"return_execution_time","repo":"bark-simulator/bark","repo_kind":"official","path":"bark/runtime/commons/commons.py","file_url":"https://github.com/bark-simulator/bark/blob/HEAD/bark/runtime/commons/commons.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2cbc7b7a886e6b50"}},{"code_sha256_prefix":"4ce488f0946799a8","entry":"serialize_benchmark_config","repo":"bark-simulator/bark","repo_kind":"official","path":"bark/benchmark/benchmark_runner_mp.py","file_url":"https://github.com/bark-simulator/bark/blob/HEAD/bark/benchmark/benchmark_runner_mp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ce488f0946799a8"}},{"code_sha256_prefix":"5057a516509b52b5","entry":"serialize_scenario","repo":"bark-simulator/bark","repo_kind":"official","path":"bark/benchmark/benchmark_runner_mp.py","file_url":"https://github.com/bark-simulator/bark/blob/HEAD/bark/benchmark/benchmark_runner_mp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5057a516509b52b5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}