{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2604-23716","title":"Information-Theoretic Measures in AI: A Practical Decision Framework","arxiv_id":"2604.23716","date":"2026-04-26","proceeding":null,"authors":["Nikolaos Al. Papadopoulos","Konstantinos E. Psannis"],"abstract":"Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation learning and feature selection, and transfer entropy reveals directed influence in dynamical systems. Despite wide adoption, measure selection is often decoupled from estimator assumptions, failure modes, and safe inferential claims. This survey provides a practical decision framework for four foundational measures - Entropy, KL divergence/cross-entropy, Mutual Information, and Transfer Entropy - organized around three prescriptive questions for each: (i) what question does the measure answer and in which AI context; (ii) which estimator is appropriate for the data type and dimensionality; and (iii) what is the most dangerous misuse. The framework is operationalized in two complementary artifacts: a measure-selection flowchart and a master decision table. We cover both AI/ML and decision-making agent application domains per measure, with standardized Bridge notes linking IT quantities to cognitive and neuroscientific constructs. Two worked examples illustrate the framework on concrete practitioner scenarios spanning representation learning and temporal influence analysis, and a reproducible multi-agent case study across three learning architectures validates the transfer-entropy surrogate-testing guardrail against a null control.","url_abs":"https://arxiv.org/abs/2604.23716","url_pdf":"https://arxiv.org/pdf/2604.23716","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2604.23716","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2604.23716"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/dentros/gamebrains","reach":null}],"summary":{"ran_fixture":4,"ran_draft_wrong":7},"by_repo_kind":{"found_in_text":{"samples":11,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"eb9bfdef141c6159","entry":"_entropy_bits_mm","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"eb9bfdef141c6159"}},{"code_sha256_prefix":"3d995975f57475ed","entry":"_mi_bits","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"3d995975f57475ed"}},{"code_sha256_prefix":"9832f39c8b8368bf","entry":"_reconstruct_observations","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"9832f39c8b8368bf"}},{"code_sha256_prefix":"e30741a503f6d077","entry":"_te_bits","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"e30741a503f6d077"}},{"code_sha256_prefix":"bb0dc3561adc98ef","entry":"compute_all","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"bb0dc3561adc98ef"}},{"code_sha256_prefix":"d4a7e08ed74aecce","entry":"mutual_information_per_agent","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"d4a7e08ed74aecce"}},{"code_sha256_prefix":"4396df363a697c95","entry":"predictive_information_per_agent","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"4396df363a697c95"}},{"code_sha256_prefix":"3ebd270f0a46aded","entry":"stationarity_report","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"3ebd270f0a46aded"}},{"code_sha256_prefix":"2bd7196a532e7f73","entry":"suggested_burn_in","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"2bd7196a532e7f73"}},{"code_sha256_prefix":"2edf4c06649663b7","entry":"transfer_entropy_pairwise","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"2edf4c06649663b7"}},{"code_sha256_prefix":"285a85ea051fa5ea","entry":"window_is_trustworthy","repo":"dentros/gamebrains","repo_kind":"found_in_text","path":"metrics/information.py","file_url":"https://github.com/dentros/gamebrains/blob/HEAD/metrics/information.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"285a85ea051fa5ea"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}