{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2511-19359","title":"Enhancing Conformal Prediction via Class Similarity","arxiv_id":"2511.19359","date":"2025-11-24","proceeding":"ICML","authors":["Ariel Fargion","Lahav Dabah","Tom Tirer"],"abstract":"Conformal Prediction (CP) has emerged as a powerful statistical framework for high-stakes classification applications. Instead of predicting a single class, CP generates a prediction set, guaranteed to include the true label with a pre-specified probability. The performance of different CP methods is typically assessed by their average prediction set size. In setups where the classes can be partitioned into semantic groups, e.g., diseases that require similar treatment, users can benefit from prediction sets that are not only small on average, but also contain a small number of semantically different groups. This paper begins by addressing this problem and ultimately offers a widely applicable tool for boosting any CP method on any dataset. First, given a class partition, we propose augmenting the CP score function with a term that penalizes predictions with out-of-group errors. We theoretically analyze this strategy and prove its advantages for group-related metrics. Surprisingly, we show mathematically that, for common class partitions, it can also reduce the average set size of any CP score function. Our analysis reveals the class-similarity factors behind this improvement and motivates a variant that can further reduce prediction set size by leveraging the model's embeddings, without requiring any human semantic partition. Finally, we present an extensive empirical study, encompassing prominent CP methods, multiple models, and several datasets, which demonstrates that our class-similarity-based approach consistently enhances CP methods.","url_abs":"https://arxiv.org/abs/2511.19359","url_pdf":"https://arxiv.org/pdf/2511.19359","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2511.19359","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2511.19359"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ariel361/CP_via_CS","reach":null}],"summary":{"ran_violates":1,"ran_fixture":1,"unverified":4},"by_repo_kind":{"found_in_text":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"f40a4e1a543366f9","entry":"build_reg_vec","repo":"ariel361/CP_via_CS","repo_kind":"found_in_text","path":"src/score_functions.py","file_url":"https://github.com/ariel361/CP_via_CS/blob/HEAD/src/score_functions.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f40a4e1a543366f9"}},{"code_sha256_prefix":"70ebe01fda2b2a97","entry":"raps_cal_scores","repo":"ariel361/CP_via_CS","repo_kind":"found_in_text","path":"src/score_functions.py","file_url":"https://github.com/ariel361/CP_via_CS/blob/HEAD/src/score_functions.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70ebe01fda2b2a97"}},{"code_sha256_prefix":"d5209caea1c50d7e","entry":"conformal_quantile","repo":"ariel361/CP_via_CS","repo_kind":"found_in_text","path":"src/score_functions.py","file_url":"https://github.com/ariel361/CP_via_CS/blob/HEAD/src/score_functions.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d5209caea1c50d7e"}},{"code_sha256_prefix":"671679146af5d9a1","entry":"run_raps_baseline","repo":"ariel361/CP_via_CS","repo_kind":"found_in_text","path":"src/experiments_original.py","file_url":"https://github.com/ariel361/CP_via_CS/blob/HEAD/src/experiments_original.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"671679146af5d9a1"}},{"code_sha256_prefix":"44b33415594e6724","entry":"run_raps_ma_avg_opt","repo":"ariel361/CP_via_CS","repo_kind":"found_in_text","path":"src/experiments_original.py","file_url":"https://github.com/ariel361/CP_via_CS/blob/HEAD/src/experiments_original.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"44b33415594e6724"}},{"code_sha256_prefix":"3481042569c9f722","entry":"run_raps_ma_binary_dist_avg_opt","repo":"ariel361/CP_via_CS","repo_kind":"found_in_text","path":"src/experiments_original.py","file_url":"https://github.com/ariel361/CP_via_CS/blob/HEAD/src/experiments_original.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3481042569c9f722"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_api"},"syntology_extracted_results":null}