{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimal-and-greedy-algorithms-for-multi-armed","title":"The Unreasonable Effectiveness of Greedy Algorithms in Multi-Armed Bandit with Many Arms","arxiv_id":"2002.10121","date":"2020-02-24","proceeding":null,"authors":["Mohsen Bayati","Nima Hamidi","Ramesh Johari","Khashayar Khosravi"],"abstract":"We investigate a Bayesian $k$-armed bandit problem in the \\emph{many-armed} regime, where $k \\geq \\sqrt{T}$ and $T$ represents the time horizon. Initially, and aligned with recent literature on many-armed bandit problems, we observe that subsampling plays a key role in designing optimal algorithms; the conventional UCB algorithm is sub-optimal, whereas a subsampled UCB (SS-UCB), which selects $\\Theta(\\sqrt{T})$ arms for execution under the UCB framework, achieves rate-optimality. However, despite SS-UCB's theoretical promise of optimal regret, it empirically underperforms compared to a greedy algorithm that consistently chooses the empirically best arm. This observation extends to contextual settings through simulations with real-world data. Our findings suggest a new form of \\emph{free exploration} beneficial to greedy algorithms in the many-armed context, fundamentally linked to a tail event concerning the prior distribution of arm rewards. This finding diverges from the notion of free exploration, which relates to covariate variation, as recently discussed in contextual bandit literature. Expanding upon these insights, we establish that the subsampled greedy approach not only achieves rate-optimality for Bernoulli bandits within the many-armed regime but also attains sublinear regret across broader distributions. Collectively, our research indicates that in the many-armed regime, practitioners might find greater value in adopting greedy algorithms.","url_abs":"https://arxiv.org/abs/2002.10121v4","url_pdf":"https://arxiv.org/pdf/2002.10121v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimal-and-greedy-algorithms-for-multi-armed","repo_url":"https://github.com/khashayarkhv/many-armed-bandit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"optimal-and-greedy-algorithms-for-multi-armed","repo_url":"https://github.com/jehankairasvakharia/Santa_2020_Jehan","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"multi-armed-bandits","task_name":"Multi-Armed Bandits"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2002.10121","atlas_url":"https://app.syntology.ai/?focus=2002.10121","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2002.10121"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/khashayarkhv/many-armed-bandit","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jehankairasvakharia/Santa_2020_Jehan","reach":{"status":"ok"}}],"summary":{"ran_violates":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"75362415986f6b4c","entry":"make_rgb_transparent","repo":"khashayarkhv/many-armed-bandit","repo_kind":"official","path":"montecarlo.py","file_url":"https://github.com/khashayarkhv/many-armed-bandit/blob/HEAD/montecarlo.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"75362415986f6b4c"}},{"code_sha256_prefix":"03c2c26294737976","entry":"run_montecarlo","repo":"khashayarkhv/many-armed-bandit","repo_kind":"official","path":"montecarlo.py","file_url":"https://github.com/khashayarkhv/many-armed-bandit/blob/HEAD/montecarlo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03c2c26294737976"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}