{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generalization-bounds-via-conditional-f","title":"Generalization Bounds via Conditional $f$-Information","arxiv_id":"2410.22887","date":"2024-10-30","proceeding":null,"authors":["Ziqiao Wang","Yongyi Mao"],"abstract":"In this work, we introduce novel information-theoretic generalization bounds using the conditional $f$-information framework, an extension of the traditional conditional mutual information (MI) framework. We provide a generic approach to derive generalization bounds via $f$-information in the supersample setting, applicable to both bounded and unbounded loss functions. Unlike previous MI-based bounds, our proof strategy does not rely on upper bounding the cumulant-generating function (CGF) in the variational formula of MI. Instead, we set the CGF or its upper bound to zero by carefully selecting the measurable function invoked in the variational formula. Although some of our techniques are partially inspired by recent advances in the coin-betting framework (e.g., Jang et al. (2023)), our results are independent of any previous findings from regret guarantees of online gambling algorithms. Additionally, our newly derived MI-based bound recovers many previous results and improves our understanding of their potential limitations. Finally, we empirically compare various $f$-information measures for generalization, demonstrating the improvement of our new bounds over the previous bounds.","url_abs":"https://arxiv.org/abs/2410.22887v1","url_pdf":"https://arxiv.org/pdf/2410.22887v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generalization-bounds-via-conditional-f","repo_url":"https://github.com/ZiqiaoWangGeothe/Conditional-f-Information-Bound","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"generalization-bounds","task_name":"Generalization Bounds"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.22887","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.22887"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ZiqiaoWangGeothe/Conditional-f-Information-Bound","reach":null}],"summary":{"ran_honours":2,"ran_fixture":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"db64abb644176531","entry":"cifar_resnet50_ld_schedule","repo":"ziqiaowanggeothe/conditional-f-information-bound","repo_kind":"official","path":"scripts/fcmi_train_classifier.py","file_url":"https://github.com/ziqiaowanggeothe/conditional-f-information-bound/blob/HEAD/scripts/fcmi_train_classifier.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"db64abb644176531"}},{"code_sha256_prefix":"f1e8556c82281e71","entry":"compute_error","repo":"ziqiaowanggeothe/conditional-f-information-bound","repo_kind":"official","path":"fcmi_parse_results.py","file_url":"https://github.com/ziqiaowanggeothe/conditional-f-information-bound/blob/HEAD/fcmi_parse_results.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f1e8556c82281e71"}},{"code_sha256_prefix":"ab716744c977f474","entry":"discrete_mi_est","repo":"ZiqiaoWangGeothe/Conditional-f-Information-Bound","repo_kind":"official","path":"modules/bound_utils.py","file_url":"https://github.com/ZiqiaoWangGeothe/Conditional-f-Information-Bound/blob/HEAD/modules/bound_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab716744c977f474"}},{"code_sha256_prefix":"e3d10ba79eb18471","entry":"estimate_fcmi_bound_classification","repo":"ZiqiaoWangGeothe/Conditional-f-Information-Bound","repo_kind":"official","path":"modules/bound_utils.py","file_url":"https://github.com/ZiqiaoWangGeothe/Conditional-f-Information-Bound/blob/HEAD/modules/bound_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e3d10ba79eb18471"}},{"code_sha256_prefix":"32ebe92e287f1e8c","entry":"mnist_ld_schedule","repo":"ziqiaowanggeothe/conditional-f-information-bound","repo_kind":"official","path":"scripts/fcmi_train_classifier.py","file_url":"https://github.com/ziqiaowanggeothe/conditional-f-information-bound/blob/HEAD/scripts/fcmi_train_classifier.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"32ebe92e287f1e8c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}