{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mitigating-boundary-ambiguity-and-inherent","title":"Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models","arxiv_id":"2406.07001","date":"2024-06-11","proceeding":null,"authors":["Zhenyi Lu","Jie Tian","Wei Wei","Xiaoye Qu","Yu Cheng","Wenfeng Xie","Dangyang Chen"],"abstract":"Text classification is a crucial task encountered frequently in practical scenarios, yet it is still under-explored in the era of large language models (LLMs). This study shows that LLMs are vulnerable to changes in the number and arrangement of options in text classification. Our extensive empirical analyses reveal that the key bottleneck arises from ambiguous decision boundaries and inherent biases towards specific tokens and positions. To mitigate these issues, we make the first attempt and propose a novel two-stage classification framework for LLMs. Our approach is grounded in the empirical observation that pairwise comparisons can effectively alleviate boundary ambiguity and inherent bias. Specifically, we begin with a self-reduction technique to efficiently narrow down numerous options, which contributes to reduced decision space and a faster comparison process. Subsequently, pairwise contrastive comparisons are employed in a chain-of-thought manner to draw out nuances and distinguish confusable options, thus refining the ambiguous decision boundary. Extensive experiments on four datasets (Banking77, HWU64, LIU54, and Clinic150) verify the effectiveness of our framework. Furthermore, benefitting from our framework, various LLMs can achieve consistent improvements. Our code and data are available in \\url{https://github.com/Chuge0335/PC-CoT}.","url_abs":"https://arxiv.org/abs/2406.07001v1","url_pdf":"https://arxiv.org/pdf/2406.07001v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mitigating-boundary-ambiguity-and-inherent","repo_url":"https://github.com/chuge0335/pc-cot","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2406.07001","atlas_url":"https://app.syntology.ai/?focus=2406.07001","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.07001"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chuge0335/pc-cot","reach":{"status":"ok"}}],"summary":{"ran":13,"unverified":2},"by_repo_kind":{"official":{"samples":15,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"c0eb035295701366","entry":"calculate_similarity","repo":"chuge0335/pc-cot","repo_kind":"official","path":"prompter.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/prompter.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c0eb035295701366"}},{"code_sha256_prefix":"4e62f2cae9acd638","entry":"conver","repo":"chuge0335/pc-cot","repo_kind":"official","path":"dataloader.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4e62f2cae9acd638"}},{"code_sha256_prefix":"a519bacd972c0be9","entry":"extract_format","repo":"chuge0335/pc-cot","repo_kind":"official","path":"extract_answer.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/extract_answer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a519bacd972c0be9"}},{"code_sha256_prefix":"7f6ef51c414f4f25","entry":"extract_list","repo":"chuge0335/pc-cot","repo_kind":"official","path":"extract_answer.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/extract_answer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f6ef51c414f4f25"}},{"code_sha256_prefix":"7b88d8c2a68c7e24","entry":"find_least_similar_pair","repo":"chuge0335/pc-cot","repo_kind":"official","path":"prompter.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/prompter.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b88d8c2a68c7e24"}},{"code_sha256_prefix":"f7289b4621cb8dfe","entry":"get_parameters","repo":"chuge0335/pc-cot","repo_kind":"official","path":"construct_prompt.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/construct_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f7289b4621cb8dfe"}},{"code_sha256_prefix":"a38d37839ce6f005","entry":"infer","repo":"chuge0335/pc-cot","repo_kind":"official","path":"openai_client.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/openai_client.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a38d37839ce6f005"}},{"code_sha256_prefix":"c09b5d14f51af2ba","entry":"most_common_elements","repo":"chuge0335/pc-cot","repo_kind":"official","path":"label_select.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/label_select.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c09b5d14f51af2ba"}},{"code_sha256_prefix":"a3f1cee125771755","entry":"pwd","repo":"chuge0335/pc-cot","repo_kind":"official","path":"utils.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a3f1cee125771755"}},{"code_sha256_prefix":"24dbe2252563041a","entry":"sort_fun","repo":"chuge0335/pc-cot","repo_kind":"official","path":"extract_answer.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/extract_answer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"24dbe2252563041a"}},{"code_sha256_prefix":"6f92b5a5a9bc5643","entry":"split_multi_round","repo":"chuge0335/pc-cot","repo_kind":"official","path":"construct_prompt.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/construct_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6f92b5a5a9bc5643"}},{"code_sha256_prefix":"4085c4d168961756","entry":"top_acc","repo":"chuge0335/pc-cot","repo_kind":"official","path":"utils.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4085c4d168961756"}},{"code_sha256_prefix":"eda4f1b5a754ad75","entry":"valid","repo":"chuge0335/pc-cot","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eda4f1b5a754ad75"}},{"code_sha256_prefix":"ac470f6ea3fe003f","entry":"new_sample","repo":"chuge0335/pc-cot","repo_kind":"official","path":"utils.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac470f6ea3fe003f"}},{"code_sha256_prefix":"130d80d6ceacd106","entry":"split_data","repo":"chuge0335/pc-cot","repo_kind":"official","path":"openai_client.py","file_url":"https://github.com/chuge0335/pc-cot/blob/HEAD/openai_client.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"130d80d6ceacd106"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}