{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/retrieval-style-in-context-learning-for-few","title":"Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classification","arxiv_id":"2406.17534","date":"2024-06-25","proceeding":null,"authors":["Huiyao Chen","Yu Zhao","Zulong Chen","Mengjia Wang","Liangyue Li","Meishan Zhang","Min Zhang"],"abstract":"Hierarchical text classification (HTC) is an important task with broad applications, while few-shot HTC has gained increasing interest recently. While in-context learning (ICL) with large language models (LLMs) has achieved significant success in few-shot learning, it is not as effective for HTC because of the expansive hierarchical label sets and extremely-ambiguous labels. In this work, we introduce the first ICL-based framework with LLM for few-shot HTC. We exploit a retrieval database to identify relevant demonstrations, and an iterative policy to manage multi-layer hierarchical labels. Particularly, we equip the retrieval database with HTC label-aware representations for the input texts, which is achieved by continual training on a pretrained language model with masked language modeling (MLM), layer-wise classification (CLS, specifically for HTC), and a novel divergent contrastive learning (DCL, mainly for adjacent semantically-similar labels) objective. Experimental results on three benchmark datasets demonstrate superior performance of our method, and we can achieve state-of-the-art results in few-shot HTC.","url_abs":"https://arxiv.org/abs/2406.17534v2","url_pdf":"https://arxiv.org/pdf/2406.17534v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"retrieval-style-in-context-learning-for-few","repo_url":"https://github.com/DreamH1gh/TACL2024","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"few-shot-htc","task_name":"Few-shot HTC"},{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"masked-language-modeling","task_name":"Masked Language Modeling"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"few-shot-htc-2","task_name":"few-shot-htc"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"fpn","method_name":"FPN"},{"method_slug":"htc","method_name":"HTC"},{"method_slug":"rpn","method_name":"RPN"},{"method_slug":"roi-align","method_name":"RoIAlign"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.17534","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}