{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-19370","title":"Human-like autonomy emerges from self-play and a pinch of human data","arxiv_id":"2606.19370","date":"2026-06-11","proceeding":null,"authors":["Daphne Cornelisse","Julian Hunt","Zixu Zhang","Waël Doulazmi","Kevin Joseph","Jaime Fernández Fisac","Eugene Vinitsky"],"abstract":"Self-play reinforcement learning has recently emerged as a way to train driving policies without any human data. It uses cheap, large-scale simulations to substitute expensive, large-scale human driving demonstrations. A key limitation of this approach is that policies trained through pure self-play can learn effective but alien driving conventions incompatible with people. Previous works attempt to mitigate such behavioral misalignments through extensive reward engineering and domain randomization, which are brittle and labor-intensive. Instead of completely discarding human demonstrations, our method treats them as a regularization objective on top of a minimal safe goal-reaching reward. Like the spice in a good stew, we find that a little human data goes a long way: our method uses only 30 minutes of human demonstrations, 2500x fewer than comparable imitation learning approaches. Resulting policies coordinate with held-out human trajectories and complete training in 15 hours on a single consumer-grade GPU. Videos and full source code are available at https://spiced-self-play.com/.","url_abs":"https://arxiv.org/abs/2606.19370","url_pdf":"https://arxiv.org/pdf/2606.19370","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.19370","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.19370"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/NVlabs/catk","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"595b1e0da839016f","entry":"get_agent_features","repo":"NVlabs/catk","repo_kind":"found_in_text","path":"src/data_preprocess.py","file_url":"https://github.com/NVlabs/catk/blob/HEAD/src/data_preprocess.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"595b1e0da839016f"}},{"code_sha256_prefix":"0aaffe2584fa0de6","entry":"get_scenario_id_int_tensor","repo":"NVlabs/catk","repo_kind":"found_in_text","path":"src/utils/wosac_utils.py","file_url":"https://github.com/NVlabs/catk/blob/HEAD/src/utils/wosac_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"0aaffe2584fa0de6"}},{"code_sha256_prefix":"62659e55e98f3487","entry":"get_traffic_light_features","repo":"NVlabs/catk","repo_kind":"found_in_text","path":"src/utils/vis_waymo.py","file_url":"https://github.com/NVlabs/catk/blob/HEAD/src/utils/vis_waymo.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"62659e55e98f3487"}},{"code_sha256_prefix":"fc3d256ed49bb1af","entry":"get_map_features","repo":"NVlabs/catk","repo_kind":"found_in_text","path":"src/data_preprocess.py","file_url":"https://github.com/NVlabs/catk/blob/HEAD/src/data_preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fc3d256ed49bb1af"}},{"code_sha256_prefix":"d5afdfd28e2e6bc3","entry":"get_map_features","repo":"NVlabs/catk","repo_kind":"found_in_text","path":"src/utils/vis_waymo.py","file_url":"https://github.com/NVlabs/catk/blob/HEAD/src/utils/vis_waymo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d5afdfd28e2e6bc3"}},{"code_sha256_prefix":"f192d45022a9cf75","entry":"process_dynamic_map","repo":"NVlabs/catk","repo_kind":"found_in_text","path":"src/data_preprocess.py","file_url":"https://github.com/NVlabs/catk/blob/HEAD/src/data_preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f192d45022a9cf75"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}