{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/value-alignment-verification","title":"Value Alignment Verification","arxiv_id":"2012.01557","date":"2020-12-02","proceeding":null,"authors":["Daniel S. Brown","Jordan Schneider","Anca D. Dragan","Scott Niekum"],"abstract":"As humans interact with autonomous agents to perform increasingly complicated, potentially risky tasks, it is important to be able to efficiently evaluate an agent's performance and correctness. In this paper we formalize and theoretically analyze the problem of efficient value alignment verification: how to efficiently test whether the behavior of another agent is aligned with a human's values. The goal is to construct a kind of \"driver's test\" that a human can give to any agent which will verify value alignment via a minimal number of queries. We study alignment verification problems with both idealized humans that have an explicit reward function as well as problems where they have implicit values. We analyze verification of exact value alignment for rational agents and propose and analyze heuristic and approximate value alignment verification tests in a wide range of gridworlds and a continuous autonomous driving domain. Finally, we prove that there exist sufficient conditions such that we can verify exact and approximate alignment across an infinite set of test environments via a constant-query-complexity alignment test.","url_abs":"https://arxiv.org/abs/2012.01557v2","url_pdf":"https://arxiv.org/pdf/2012.01557v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"value-alignment-verification","repo_url":"https://github.com/dsbrown1331/vav-icml","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2012.01557","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2012.01557"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dsbrown1331/vav-icml","reach":null}],"summary":{"ran_draft_wrong":1,"ran_fixture":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8c46d08023023f82","entry":"generate_psi","repo":"dsbrown1331/vav-icml","repo_kind":"official","path":"drivingsim_vav/algos.py","file_url":"https://github.com/dsbrown1331/vav-icml/blob/HEAD/drivingsim_vav/algos.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c46d08023023f82"}},{"code_sha256_prefix":"7211ac6cf9bb9e60","entry":"information_objective_psi","repo":"dsbrown1331/vav-icml","repo_kind":"official","path":"drivingsim_vav/algos.py","file_url":"https://github.com/dsbrown1331/vav-icml/blob/HEAD/drivingsim_vav/algos.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7211ac6cf9bb9e60"}},{"code_sha256_prefix":"c63e702a58807835","entry":"volume_objective_psi","repo":"dsbrown1331/vav-icml","repo_kind":"official","path":"drivingsim_vav/algos.py","file_url":"https://github.com/dsbrown1331/vav-icml/blob/HEAD/drivingsim_vav/algos.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c63e702a58807835"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}