{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-universal-predictors","title":"Learning Universal Predictors","arxiv_id":"2401.14953","date":"2024-01-26","proceeding":null,"authors":["Jordi Grau-Moya","Tim Genewein","Marcus Hutter","Laurent Orseau","Grégoire Delétang","Elliot Catt","Anian Ruoss","Li Kevin Wenliang","Christopher Mattern","Matthew Aitchison","Joel Veness"],"abstract":"Meta-learning has emerged as a powerful approach to train neural networks to learn new tasks quickly from limited data. Broad exposure to different tasks leads to versatile representations enabling general problem solving. But, what are the limits of meta-learning? In this work, we explore the potential of amortizing the most powerful universal predictor, namely Solomonoff Induction (SI), into neural networks via leveraging meta-learning to its limits. We use Universal Turing Machines (UTMs) to generate training data used to expose networks to a broad range of patterns. We provide theoretical analysis of the UTM data generation processes and meta-training protocols. We conduct comprehensive experiments with neural architectures (e.g. LSTMs, Transformers) and algorithmic data generators of varying complexity and universality. Our results suggest that UTM data is a valuable resource for meta-learning, and that it can be used to train neural networks capable of learning universal prediction strategies.","url_abs":"https://arxiv.org/abs/2401.14953v1","url_pdf":"https://arxiv.org/pdf/2401.14953v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-universal-predictors","repo_url":"https://github.com/google-deepmind/neural_networks_solomonoff_induction","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"meta-learning","task_name":"Meta-Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.14953","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.14953"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-deepmind/neural_networks_solomonoff_induction","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f915c59353ea23e2","entry":"sinusoid_position_encoding","repo":"google-deepmind/neural_networks_solomonoff_induction","repo_kind":"official","path":"models/transformer.py","file_url":"https://github.com/google-deepmind/neural_networks_solomonoff_induction/blob/HEAD/models/transformer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f915c59353ea23e2"}},{"code_sha256_prefix":"ee725f97222e1399","entry":"embed_sequences","repo":"google-deepmind/neural_networks_solomonoff_induction","repo_kind":"official","path":"models/transformer.py","file_url":"https://github.com/google-deepmind/neural_networks_solomonoff_induction/blob/HEAD/models/transformer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ee725f97222e1399"}},{"code_sha256_prefix":"a145acc7af542d06","entry":"layer_norm","repo":"google-deepmind/neural_networks_solomonoff_induction","repo_kind":"official","path":"models/transformer.py","file_url":"https://github.com/google-deepmind/neural_networks_solomonoff_induction/blob/HEAD/models/transformer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a145acc7af542d06"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}