{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gipcol-graph-injected-soft-prompting-for","title":"GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning","arxiv_id":"2311.05729","date":"2023-11-09","proceeding":null,"authors":["Guangyue Xu","Joyce Chai","Parisa Kordjamshidi"],"abstract":"Pre-trained vision-language models (VLMs) have achieved promising success in many fields, especially with prompt learning paradigm. In this work, we propose GIP-COL (Graph-Injected Soft Prompting for COmpositional Learning) to better explore the compositional zero-shot learning (CZSL) ability of VLMs within the prompt-based learning framework. The soft prompt in GIPCOL is structured and consists of the prefix learnable vectors, attribute label and object label. In addition, the attribute and object labels in the soft prompt are designated as nodes in a compositional graph. The compositional graph is constructed based on the compositional structure of the objects and attributes extracted from the training data and consequently feeds the updated concept representation into the soft prompt to capture this compositional structure for a better prompting for CZSL. With the new prompting strategy, GIPCOL achieves state-of-the-art AUC results on all three CZSL benchmarks, including MIT-States, UT-Zappos, and C-GQA datasets in both closed and open settings compared to previous non-CLIP as well as CLIP-based methods. We analyze when and why GIPCOL operates well given the CLIP backbone and its training data limitations, and our findings shed light on designing more effective prompts for CZSL","url_abs":"https://arxiv.org/abs/2311.05729v1","url_pdf":"https://arxiv.org/pdf/2311.05729v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gipcol-graph-injected-soft-prompting-for","repo_url":"https://github.com/hlr/gipcol","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"compositional-zero-shot-learning","task_name":"Compositional Zero-Shot Learning"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.05729","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.05729"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hlr/gipcol","reach":{"status":"ok"}}],"summary":{"ran":7,"unverified":5},"by_repo_kind":{"official":{"samples":12,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"a23cf5bf1e4b1696","entry":"build_lr_scheduler","repo":"hlr/gipcol","repo_kind":"official","path":"Models/CoopModel/Scheduler/lr_scheduler.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/Models/CoopModel/Scheduler/lr_scheduler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a23cf5bf1e4b1696"}},{"code_sha256_prefix":"b9bd2410ef8caf6a","entry":"build_optimizer_from_coop","repo":"hlr/gipcol","repo_kind":"official","path":"Models/CoopModel/Optimizer/optimizer.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/Models/CoopModel/Optimizer/optimizer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b9bd2410ef8caf6a"}},{"code_sha256_prefix":"f104560916c04c5d","entry":"compute_cosine_similarity","repo":"hlr/gipcol","repo_kind":"official","path":"DataSet/compute_feasibility.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/DataSet/compute_feasibility.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f104560916c04c5d"}},{"code_sha256_prefix":"3d6ee201ac8ba1db","entry":"compute_representations","repo":"hlr/gipcol","repo_kind":"official","path":"script_shared_evaluator_from_csp.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/script_shared_evaluator_from_csp.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d6ee201ac8ba1db"}},{"code_sha256_prefix":"dd9bca67df4ae7b8","entry":"imagenet_transform","repo":"hlr/gipcol","repo_kind":"official","path":"DataSet/AttrOpCompDataSet.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/DataSet/AttrOpCompDataSet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dd9bca67df4ae7b8"}},{"code_sha256_prefix":"9a62704a729580fc","entry":"load_glove_embeddings","repo":"hlr/gipcol","repo_kind":"official","path":"DataSet/compute_feasibility.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/DataSet/compute_feasibility.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9a62704a729580fc"}},{"code_sha256_prefix":"6bddc3dfac41ae6c","entry":"predict_logits","repo":"hlr/gipcol","repo_kind":"official","path":"script_shared_evaluator_from_csp.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/script_shared_evaluator_from_csp.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6bddc3dfac41ae6c"}},{"code_sha256_prefix":"c47aa9e5b049a11d","entry":"build_model","repo":"hlr/gipcol","repo_kind":"official","path":"Models/CoopModel/clip/model.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/Models/CoopModel/clip/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c47aa9e5b049a11d"}},{"code_sha256_prefix":"6c217fe4e4e0585d","entry":"build_optimizer","repo":"hlr/gipcol","repo_kind":"official","path":"Models/CoopModel/Optimizer/optimizer.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/Models/CoopModel/Optimizer/optimizer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6c217fe4e4e0585d"}},{"code_sha256_prefix":"8bfeb91e02c5cca9","entry":"clip_baseline","repo":"hlr/gipcol","repo_kind":"official","path":"script_shared_evaluator_from_csp.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/script_shared_evaluator_from_csp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8bfeb91e02c5cca9"}},{"code_sha256_prefix":"fbf8c0143d9c48e3","entry":"load","repo":"hlr/gipcol","repo_kind":"official","path":"Models/CoopModel/clip/clip.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/Models/CoopModel/clip/clip.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fbf8c0143d9c48e3"}},{"code_sha256_prefix":"b5a52229ba9f8344","entry":"train_model","repo":"hlr/gipcol","repo_kind":"official","path":"script_train_graph_prompt.py","file_url":"https://github.com/hlr/gipcol/blob/HEAD/script_train_graph_prompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b5a52229ba9f8344"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}