{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-improved-baseline-for-reasoning","title":"LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model","arxiv_id":"2312.17240","date":"2023-12-28","proceeding":null,"authors":["Senqiao Yang","Tianyuan Qu","Xin Lai","Zhuotao Tian","Bohao Peng","Shu Liu","Jiaya Jia"],"abstract":"While LISA effectively bridges the gap between segmentation and large language models to enable reasoning segmentation, it poses certain limitations: unable to distinguish different instances of the target region, and constrained by the pre-defined textual response formats. In this work, we introduce LISA++, an update to the existing LISA model, focusing on improving core functionalities while keeping the base architecture intact. The main enhancements in LISA++ include: \\textbf{1) Enhanced Segmentation}: The instance segmentation ability has been added, providing a more detailed scene analysis along with the existing multi-region semantic segmentation. \\textbf{2) More Natural Conversation}: Improved capability for multi-turn dialogue, with the ability to incorporate segmentation results directly into text responses, i.e., Segmentation in Dialogue (SiD). These improvements are achieved by curating the existing samples of generic segmentation datasets, aimed specifically at enhancing the segmentation and conversational skills without structural change and additional data sources. Comparative analysis with the original LISA model shows significant advancements in these areas, positioning LISA++ as a notable upgrade in visual understanding and interaction. LISA++'s adaptability and improved features highlight the versatility of the mask-as-embedding paradigm proposed by LISA, and the potential as a foundational model for diverse applications.","url_abs":"https://arxiv.org/abs/2312.17240v3","url_pdf":"https://arxiv.org/pdf/2312.17240v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-improved-baseline-for-reasoning","repo_url":"https://github.com/dvlab-research/lisa","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"reasoning-segmentation","task_name":"Reasoning Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.17240","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.17240"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dvlab-research/lisa","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":2,"ran_draft_wrong":1,"ran":1,"unverified":10},"by_repo_kind":{"listed":{"samples":14,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a1c2083ec2eaeb22","entry":"dice_loss","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/LISA.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/LISA.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a1c2083ec2eaeb22"}},{"code_sha256_prefix":"77ff24f1cbcb4f2d","entry":"preprocess","repo":"dvlab-research/lisa","repo_kind":"listed","path":"chat.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/chat.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"77ff24f1cbcb4f2d"}},{"code_sha256_prefix":"1df990c375318896","entry":"process_images","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/llava/mm_utils.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/llava/mm_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1df990c375318896"}},{"code_sha256_prefix":"a9292f5d89194794","entry":"sigmoid_ce_loss","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/LISA.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/LISA.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a9292f5d89194794"}},{"code_sha256_prefix":"3f50fb8374909f5f","entry":"build_sam_vit_b","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/segment_anything/build_sam.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/segment_anything/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3f50fb8374909f5f"}},{"code_sha256_prefix":"bf61a91555ac5280","entry":"build_sam_vit_h","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/segment_anything/build_sam.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/segment_anything/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bf61a91555ac5280"}},{"code_sha256_prefix":"e2ae4c696b8c968c","entry":"build_sam_vit_l","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/segment_anything/build_sam.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/segment_anything/build_sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e2ae4c696b8c968c"}},{"code_sha256_prefix":"245343ff311f6219","entry":"get_default_conv_template","repo":"dvlab-research/lisa","repo_kind":"listed","path":"utils/conversation.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/utils/conversation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"245343ff311f6219"}},{"code_sha256_prefix":"0dde2e782959c0bd","entry":"load_image_from_base64","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/llava/mm_utils.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/llava/mm_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0dde2e782959c0bd"}},{"code_sha256_prefix":"1496505dfa11cab2","entry":"parse_args","repo":"dvlab-research/lisa","repo_kind":"listed","path":"merge_lora_weights_and_save_hf_model.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/merge_lora_weights_and_save_hf_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1496505dfa11cab2"}},{"code_sha256_prefix":"58e2ab5845d98f4c","entry":"parse_args","repo":"dvlab-research/lisa","repo_kind":"listed","path":"chat.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/chat.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"58e2ab5845d98f4c"}},{"code_sha256_prefix":"a0d5897eb3717c5c","entry":"parse_args","repo":"dvlab-research/lisa","repo_kind":"listed","path":"train_ds.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/train_ds.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a0d5897eb3717c5c"}},{"code_sha256_prefix":"37899f22fb191b37","entry":"pretty_print_semaphore","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/llava/utils.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/llava/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"37899f22fb191b37"}},{"code_sha256_prefix":"f9939a84b9a65279","entry":"violates_moderation","repo":"dvlab-research/lisa","repo_kind":"listed","path":"model/llava/utils.py","file_url":"https://github.com/dvlab-research/lisa/blob/HEAD/model/llava/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9939a84b9a65279"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}