{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tight-regret-bounds-for-single-pass-streaming","title":"Tight Regret Bounds for Single-pass Streaming Multi-armed Bandits","arxiv_id":"2306.02208","date":"2023-06-03","proceeding":null,"authors":["Chen Wang"],"abstract":"Regret minimization in streaming multi-armed bandits (MABs) has been studied extensively in recent years. In the single-pass setting with $K$ arms and $T$ trials, a regret lower bound of $\\Omega(T^{2/3})$ has been proved for any algorithm with $o(K)$ memory (Maiti et al. [NeurIPS'21]; Agarwal at al. [COLT'22]). On the other hand, however, the previous best regret upper bound is still $O(K^{1/3} T^{2/3}\\log^{1/3}(T))$, which is achieved by the streaming implementation of the simple uniform exploration. The $O(K^{1/3}\\log^{1/3}(T))$ gap leaves the open question of the tight regret bound in the single-pass MABs with sublinear arm memory. In this paper, we answer this open problem and complete the picture of regret minimization in single-pass streaming MABs. We first improve the regret lower bound to $\\Omega(K^{1/3}T^{2/3})$ for algorithms with $o(K)$ memory, which matches the uniform exploration regret up to a logarithm factor in $T$. We then show that the $\\log^{1/3}(T)$ factor is not necessary, and we can achieve $O(K^{1/3}T^{2/3})$ regret by finding an $\\varepsilon$-best arm and committing to it in the rest of the trials. For regret minimization with high constant probability, we can apply the single-memory $\\varepsilon$-best arm algorithms in Jin et al. [ICML'21] to obtain the optimal bound. Furthermore, for the expected regret minimization, we design an algorithm with a single-arm memory that achieves $O(K^{1/3} T^{2/3}\\log(K))$ regret, and an algorithm with $O(\\log^{*}(n))$-memory with the optimal $O(K^{1/3} T^{2/3})$ regret following the $\\varepsilon$-best arm algorithm in Assadi and Wang [STOC'20]. We further tested the empirical performances of our algorithms. The simulation results show that the proposed algorithms consistently outperform the benchmark uniform exploration algorithm by a large margin, and on occasion, reduce the regret by up to 70%.","url_abs":"https://arxiv.org/abs/2306.02208v1","url_pdf":"https://arxiv.org/pdf/2306.02208v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tight-regret-bounds-for-single-pass-streaming","repo_url":"https://github.com/jhwjhw0123/streaming-regret-minimization-mabs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"multi-armed-bandits","task_name":"Multi-Armed Bandits"},{"task_slug":"open-question","task_name":"Open-Ended Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.02208","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.02208"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/jhwjhw0123/streamingregret-minimization-MABs","reach":{"status":"gone","observed_at":"2026-09-17","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jhwjhw0123/streaming-regret-minimization-mabs","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"737d6fccaf1f934b","entry":"best_among_remaining","repo":"jhwjhw0123/streaming-regret-minimization-mabs","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/jhwjhw0123/streaming-regret-minimization-mabs/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"737d6fccaf1f934b"}},{"code_sha256_prefix":"d511e690f9e5720b","entry":"naive_eps_best_algorithm","repo":"jhwjhw0123/streaming-regret-minimization-mabs","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/jhwjhw0123/streaming-regret-minimization-mabs/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d511e690f9e5720b"}},{"code_sha256_prefix":"57a40d97fadbe7b3","entry":"uniform_exploration_algorithm","repo":"jhwjhw0123/streaming-regret-minimization-mabs","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/jhwjhw0123/streaming-regret-minimization-mabs/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57a40d97fadbe7b3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}