{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sequential-estimation-of-quantiles-with","title":"Sequential estimation of quantiles with applications to A/B-testing and best-arm identification","arxiv_id":"1906.09712","date":"2019-06-24","proceeding":null,"authors":["Steven R. Howard","Aaditya Ramdas"],"abstract":"We propose confidence sequences -- sequences of confidence intervals which are valid uniformly over time -- for quantiles of any distribution over a complete, fully-ordered set, based on a stream of i.i.d. observations. We give methods both for tracking a fixed quantile and for tracking all quantiles simultaneously. Specifically, we provide explicit expressions with small constants for intervals whose widths shrink at the fastest possible $\\sqrt{t^{-1} \\log\\log t}$ rate, along with a non-asymptotic concentration inequality for the empirical distribution function which holds uniformly over time with the same rate. The latter strengthens Smirnov's empirical process law of the iterated logarithm and extends the Dvoretzky-Kiefer-Wolfowitz inequality to hold uniformly over time. We give a new algorithm and sample complexity bound for selecting an arm with an approximately best quantile in a multi-armed bandit framework. In simulations, our method requires fewer samples than existing methods by a factor of five to fifty.","url_abs":"https://arxiv.org/abs/1906.09712v5","url_pdf":"https://arxiv.org/pdf/1906.09712v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sequential-estimation-of-quantiles-with","repo_url":"https://github.com/gostevehoward/confseq","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"sequential-estimation-of-quantiles-with","repo_url":"https://github.com/gostevehoward/quantilecs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"sequential-estimation-of-quantiles-with","repo_url":"https://github.com/WLM1ke/poptimizer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Unlicense"}},{"paper_slug":"sequential-estimation-of-quantiles-with","repo_url":"https://github.com/wannabesmith/confseq","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":null,"task_name":"valid"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1906.09712","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1906.09712"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wannabesmith/confseq","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gostevehoward/quantilecs","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gostevehoward/confseq","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/WLM1ke/poptimizer","reach":{"status":"ok","spdx":"Unlicense"}}],"summary":{"ran_fixture":2,"unverified":4},"by_repo_kind":{"listed":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"1b969d6c5e72286a","entry":"check_approx_equal","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"1b969d6c5e72286a"}},{"code_sha256_prefix":"bb009376bd6e9abd","entry":"form_predictions","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"bb009376bd6e9abd"}},{"code_sha256_prefix":"652d4d1c4e65a97f","entry":"get_root_error","repo":"WLM1ke/poptimizer","repo_kind":"listed","path":"poptimizer/core/errors.py","file_url":"https://github.com/WLM1ke/poptimizer/blob/HEAD/poptimizer/core/errors.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"652d4d1c4e65a97f"}},{"code_sha256_prefix":"eff60c9fec6ef025","entry":"init","repo":"WLM1ke/poptimizer","repo_kind":"listed","path":"poptimizer/adapters/logger.py","file_url":"https://github.com/WLM1ke/poptimizer/blob/HEAD/poptimizer/adapters/logger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"eff60c9fec6ef025"}},{"code_sha256_prefix":"a8924b4b58608b43","entry":"sorted_by_day_validator","repo":"WLM1ke/poptimizer","repo_kind":"listed","path":"poptimizer/core/domain.py","file_url":"https://github.com/WLM1ke/poptimizer/blob/HEAD/poptimizer/core/domain.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"a8924b4b58608b43"}},{"code_sha256_prefix":"7fbee07374604772","entry":"sorted_with_ticker_field_validator","repo":"WLM1ke/poptimizer","repo_kind":"listed","path":"poptimizer/core/domain.py","file_url":"https://github.com/WLM1ke/poptimizer/blob/HEAD/poptimizer/core/domain.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Unlicense","inline_ok":true,"mcp_get_code":{"code_sha256":"7fbee07374604772"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}