{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adaptive-foundation-models-for-online","title":"Scalable Exploration via Ensemble++","arxiv_id":"2407.13195","date":"2024-07-18","proceeding":null,"authors":["Yingru Li","Jiawei Xu","Baoxiang Wang","Zhi-Quan Luo"],"abstract":"Thompson Sampling is a principled method for balancing exploration and exploitation, but its real-world adoption faces computational challenges in large-scale or non-conjugate settings. While ensemble-based approaches offer partial remedies, they typically require prohibitively large ensemble sizes. We propose Ensemble++, a scalable exploration framework using a novel shared-factor ensemble architecture with random linear combinations. For linear bandits, we provide theoretical guarantees showing that Ensemble++ achieves regret comparable to exact Thompson Sampling with only $\\Theta(d \\log T)$ ensemble sizes--significantly outperforming prior methods. Crucially, this efficiency holds across both compact and finite action sets with either time-invariant or time-varying contexts without configuration changes. We extend this theoretical foundation to nonlinear rewards by replacing fixed features with learnable neural representations while preserving the same incremental update principle, effectively bridging theory and practice for real-world tasks. Comprehensive experiments across linear, quadratic, neural, and GPT-based contextual bandits validate our theoretical findings and demonstrate Ensemble++'s superior regret-computation tradeoff versus state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2407.13195v5","url_pdf":"https://arxiv.org/pdf/2407.13195v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adaptive-foundation-models-for-online","repo_url":"https://github.com/szrlee/GPT-HyperAgent","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"adaptive-foundation-models-for-online","repo_url":"https://github.com/szrlee/ensemble_plus_plus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"efficient-exploration","task_name":"Efficient Exploration"},{"task_slug":"multi-armed-bandits","task_name":"Multi-Armed Bandits"},{"task_slug":"sequential-decision-making","task_name":"Sequential Decision Making"},{"task_slug":"thompson-sampling","task_name":"Thompson Sampling"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"base","method_name":"BASE"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"discriminative-fine-tuning","method_name":"Discriminative Fine-Tuning"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt","method_name":"GPT"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.13195","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.13195"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/szrlee/GPT-HyperAgent","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/szrlee/ensemble_plus_plus","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"98e954813f8ab0d2","entry":"EnsemblePrior","repo":"szrlee/GPT-HyperAgent","repo_kind":"official","path":"network/epinet.py","file_url":"https://github.com/szrlee/GPT-HyperAgent/blob/HEAD/network/epinet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"98e954813f8ab0d2"}},{"code_sha256_prefix":"d4d0f5d0e359944d","entry":"Ensemble_exp_Layer","repo":"szrlee/ensemble_plus_plus","repo_kind":"official","path":"network/ensemble_exp.py","file_url":"https://github.com/szrlee/ensemble_plus_plus/blob/HEAD/network/ensemble_exp.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d4d0f5d0e359944d"}},{"code_sha256_prefix":"4d5766ec7b1e0a9b","entry":"EpiLinear","repo":"szrlee/GPT-HyperAgent","repo_kind":"official","path":"network/epinet.py","file_url":"https://github.com/szrlee/GPT-HyperAgent/blob/HEAD/network/epinet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4d5766ec7b1e0a9b"}},{"code_sha256_prefix":"bc22b388af547818","entry":"mlp","repo":"szrlee/ensemble_plus_plus","repo_kind":"official","path":"network/ensemble_exp.py","file_url":"https://github.com/szrlee/ensemble_plus_plus/blob/HEAD/network/ensemble_exp.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bc22b388af547818"}},{"code_sha256_prefix":"5936968885fd5884","entry":"mlp","repo":"szrlee/GPT-HyperAgent","repo_kind":"official","path":"network/epinet.py","file_url":"https://github.com/szrlee/GPT-HyperAgent/blob/HEAD/network/epinet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5936968885fd5884"}},{"code_sha256_prefix":"329391391404d253","entry":"Ensemble_exp_Linear","repo":"szrlee/ensemble_plus_plus","repo_kind":"official","path":"network/ensemble_exp.py","file_url":"https://github.com/szrlee/ensemble_plus_plus/blob/HEAD/network/ensemble_exp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"329391391404d253"}},{"code_sha256_prefix":"ee59bd8ec68d825f","entry":"Ensemble_exp_Net","repo":"szrlee/ensemble_plus_plus","repo_kind":"official","path":"network/ensemble_exp.py","file_url":"https://github.com/szrlee/ensemble_plus_plus/blob/HEAD/network/ensemble_exp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ee59bd8ec68d825f"}},{"code_sha256_prefix":"160c7eb9dff5367c","entry":"EpiNet","repo":"szrlee/GPT-HyperAgent","repo_kind":"official","path":"network/epinet.py","file_url":"https://github.com/szrlee/GPT-HyperAgent/blob/HEAD/network/epinet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"160c7eb9dff5367c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}