{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/goal-global-local-object-alignment-learning","title":"GOAL: Global-local Object Alignment Learning","arxiv_id":"2503.17782","date":"2025-03-22","proceeding":"CVPR 2025 1","authors":["Hyungyu Choi","Young Kyun Jang","Chanho Eom"],"abstract":"Vision-language models like CLIP have shown impressive capabilities in aligning images and text, but they often struggle with lengthy and detailed text descriptions because of their training focus on short and concise captions. We present GOAL (Global-local Object Alignment Learning), a novel fine-tuning method that enhances CLIP's ability to handle lengthy text by leveraging both global and local semantic alignments between image and lengthy text. Our approach consists of two key components: Local Image-Sentence Matching (LISM), which identifies corresponding pairs between image segments and descriptive sentences, and Token Similarity-based Learning (TSL), which efficiently propagates local element attention through these matched pairs. Evaluating GOAL on three new benchmarks for image-lengthy text retrieval, we demonstrate significant improvements over baseline CLIP fine-tuning, establishing a simple yet effective approach for adapting CLIP to detailed textual descriptions. Through extensive experiments, we show that our method's focus on local semantic alignment alongside global context leads to more nuanced and representative embeddings, particularly beneficial for tasks requiring fine-grained understanding of lengthy text descriptions.","url_abs":"https://arxiv.org/abs/2503.17782v1","url_pdf":"https://arxiv.org/pdf/2503.17782v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"goal-global-local-object-alignment-learning","repo_url":"https://github.com/perceptualai-lab/goal","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"descriptive","task_name":"Descriptive"},{"task_slug":"object","task_name":"Object"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"text-retrieval","task_name":"Text Retrieval"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"focus","method_name":"Focus"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2503.17782","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.17782"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/PerceptualAI-Lab/GOAL","reach":null}],"summary":{"ran_draft_wrong":1,"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"1591524e14c110fe","entry":"clip_loss","repo":"PerceptualAI-Lab/GOAL","repo_kind":"official","path":"goal.py","file_url":"https://github.com/PerceptualAI-Lab/GOAL/blob/HEAD/goal.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1591524e14c110fe"}},{"code_sha256_prefix":"a18e14dbab50e04f","entry":"get_patch_tokens_from_bbox","repo":"PerceptualAI-Lab/GOAL","repo_kind":"official","path":"goal.py","file_url":"https://github.com/PerceptualAI-Lab/GOAL/blob/HEAD/goal.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a18e14dbab50e04f"}},{"code_sha256_prefix":"16f4eb48ca758873","entry":"get_text_tokens_from_segment","repo":"PerceptualAI-Lab/GOAL","repo_kind":"official","path":"goal.py","file_url":"https://github.com/PerceptualAI-Lab/GOAL/blob/HEAD/goal.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"16f4eb48ca758873"}},{"code_sha256_prefix":"f2535872761f96c6","entry":"train","repo":"PerceptualAI-Lab/GOAL","repo_kind":"official","path":"goal.py","file_url":"https://github.com/PerceptualAI-Lab/GOAL/blob/HEAD/goal.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f2535872761f96c6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}