{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/split-kl-and-pac-bayes-split-kl-inequalities","title":"Split-kl and PAC-Bayes-split-kl Inequalities for Ternary Random Variables","arxiv_id":"2206.00706","date":"2022-06-01","proceeding":null,"authors":["Yi-Shan Wu","Yevgeny Seldin"],"abstract":"We present a new concentration of measure inequality for sums of independent bounded random variables, which we name a split-kl inequality. The inequality is particularly well-suited for ternary random variables, which naturally show up in a variety of problems, including analysis of excess losses in classification, analysis of weighted majority votes, and learning with abstention. We demonstrate that for ternary random variables the inequality is simultaneously competitive with the kl inequality, the Empirical Bernstein inequality, and the Unexpected Bernstein inequality, and in certain regimes outperforms all of them. It resolves an open question by Tolstikhin and Seldin [2013] and Mhammedi et al. [2019] on how to match simultaneously the combinatorial power of the kl inequality when the distribution happens to be close to binary and the power of Bernstein inequalities to exploit low variance when the probability mass is concentrated on the middle value. We also derive a PAC-Bayes-split-kl inequality and compare it with the PAC-Bayes-kl, PAC-Bayes-Empirical-Bennett, and PAC-Bayes-Unexpected-Bernstein inequalities in an analysis of excess losses and in an analysis of a weighted majority vote for several UCI datasets. Last but not least, our study provides the first direct comparison of the Empirical Bernstein and Unexpected Bernstein inequalities and their PAC-Bayes extensions.","url_abs":"https://arxiv.org/abs/2206.00706v2","url_pdf":"https://arxiv.org/pdf/2206.00706v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"split-kl-and-pac-bayes-split-kl-inequalities","repo_url":"https://github.com/stephanlorenzen/majorityvotebounds","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"open-question","task_name":"Open-Ended Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2206.00706","atlas_url":"https://app.syntology.ai/?focus=2206.00706","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.00706"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/stephanlorenzen/majorityvotebounds","reach":null}],"summary":{"ran_honours":4,"ran_fixture":2,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"a0a28b5650feab1c","entry":"CCPBSkl","repo":"stephanlorenzen/majorityvotebounds","repo_kind":"official","path":"mvb/bounds/ccpbskl.py","file_url":"https://github.com/stephanlorenzen/majorityvotebounds/blob/HEAD/mvb/bounds/ccpbskl.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a0a28b5650feab1c"}},{"code_sha256_prefix":"e4689aebe6ddd3f2","entry":"CCPBSkl","repo":"stephanlorenzen/majorityvotebounds","repo_kind":"official","path":"mvb/bounds/ccpbskl.py","file_url":"https://github.com/stephanlorenzen/majorityvotebounds/blob/HEAD/mvb/bounds/ccpbskl.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"e4689aebe6ddd3f2"}},{"code_sha256_prefix":"0602dac3cba67edd","entry":"Grid","repo":"stephanlorenzen/majorityvotebounds","repo_kind":"official","path":"mvb/bounds/ccpbskl.py","file_url":"https://github.com/stephanlorenzen/majorityvotebounds/blob/HEAD/mvb/bounds/ccpbskl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"0602dac3cba67edd"}},{"code_sha256_prefix":"54327cfb4ff09e1f","entry":"KL","repo":"stephanlorenzen/majorityvotebounds","repo_kind":"official","path":"mvb/bounds/ccpbskl.py","file_url":"https://github.com/stephanlorenzen/majorityvotebounds/blob/HEAD/mvb/bounds/ccpbskl.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"54327cfb4ff09e1f"}},{"code_sha256_prefix":"b255b953d7014d54","entry":"KL_binomial","repo":"stephanlorenzen/majorityvotebounds","repo_kind":"official","path":"mvb/bounds/ccpbskl.py","file_url":"https://github.com/stephanlorenzen/majorityvotebounds/blob/HEAD/mvb/bounds/ccpbskl.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b255b953d7014d54"}},{"code_sha256_prefix":"eb810c186648f0e1","entry":"solve_kl_inf","repo":"stephanlorenzen/majorityvotebounds","repo_kind":"official","path":"mvb/bounds/ccpbskl.py","file_url":"https://github.com/stephanlorenzen/majorityvotebounds/blob/HEAD/mvb/bounds/ccpbskl.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"eb810c186648f0e1"}},{"code_sha256_prefix":"4bb023801bd26616","entry":"solve_kl_sup","repo":"stephanlorenzen/majorityvotebounds","repo_kind":"official","path":"mvb/bounds/ccpbskl.py","file_url":"https://github.com/stephanlorenzen/majorityvotebounds/blob/HEAD/mvb/bounds/ccpbskl.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"4bb023801bd26616"}},{"code_sha256_prefix":"5942fac43d09bc0d","entry":"optimizeCCPBSkl","repo":"stephanlorenzen/majorityvotebounds","repo_kind":"official","path":"mvb/bounds/ccpbskl.py","file_url":"https://github.com/stephanlorenzen/majorityvotebounds/blob/HEAD/mvb/bounds/ccpbskl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"5942fac43d09bc0d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}