{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/evaluation-beyond-task-performance-analyzing","title":"Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex","arxiv_id":"2211.14673","date":"2022-11-26","proceeding":null,"authors":["Charles Lovering","Jessica Zosa Forde","George Konidaris","Ellie Pavlick","Michael L. Littman"],"abstract":"AlphaZero, an approach to reinforcement learning that couples neural networks and Monte Carlo tree search (MCTS), has produced state-of-the-art strategies for traditional board games like chess, Go, shogi, and Hex. While researchers and game commentators have suggested that AlphaZero uses concepts that humans consider important, it is unclear how these concepts are captured in the network. We investigate AlphaZero's internal representations in the game of Hex using two evaluation techniques from natural language processing (NLP): model probing and behavioral tests. In doing so, we introduce new evaluation tools to the RL community and illustrate how evaluations other than task performance can be used to provide a more complete picture of a model's strengths and weaknesses. Our analyses in the game of Hex reveal interesting patterns and generate some testable hypotheses about how such models learn in general. For example, we find that MCTS discovers concepts before the neural network learns to encode them. We also find that concepts related to short-term end-game planning are best encoded in the final layers of the model, whereas concepts related to long-term planning are encoded in the middle layers of the model.","url_abs":"https://arxiv.org/abs/2211.14673v1","url_pdf":"https://arxiv.org/pdf/2211.14673v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"evaluation-beyond-task-performance-analyzing","repo_url":"https://github.com/jzf2101/alphatology","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"board-games","task_name":"Board Games"}],"methods":[{"method_slug":"alphazero","method_name":"AlphaZero"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.14673","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.14673"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jzf2101/alphatology","reach":null}],"summary":{"ran_honours":2,"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b186ab1c8e942f64","entry":"action_to_cell","repo":"jzf2101/alphatology","repo_kind":"official","path":"boardlaw_utils.py","file_url":"https://github.com/jzf2101/alphatology/blob/HEAD/boardlaw_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b186ab1c8e942f64"}},{"code_sha256_prefix":"269f5a0c0d1832d9","entry":"cell_to_action","repo":"jzf2101/alphatology","repo_kind":"official","path":"boardlaw_utils.py","file_url":"https://github.com/jzf2101/alphatology/blob/HEAD/boardlaw_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"269f5a0c0d1832d9"}},{"code_sha256_prefix":"1509c04dc09e4781","entry":"experiment_2","repo":"jzf2101/alphatology","repo_kind":"official","path":"probing.py","file_url":"https://github.com/jzf2101/alphatology/blob/HEAD/probing.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1509c04dc09e4781"}},{"code_sha256_prefix":"08659316adfa36f0","entry":"experiment_3","repo":"jzf2101/alphatology","repo_kind":"official","path":"probing.py","file_url":"https://github.com/jzf2101/alphatology/blob/HEAD/probing.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"08659316adfa36f0"}},{"code_sha256_prefix":"5e2fc43597b32122","entry":"experiment_4","repo":"jzf2101/alphatology","repo_kind":"official","path":"probing.py","file_url":"https://github.com/jzf2101/alphatology/blob/HEAD/probing.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5e2fc43597b32122"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}