{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/segment-anything-in-high-quality","title":"Segment Anything in High Quality","arxiv_id":"2306.01567","date":"2023-06-02","proceeding":"NeurIPS 2023 11","authors":["Lei Ke","Mingqiao Ye","Martin Danelljan","Yifan Liu","Yu-Wing Tai","Chi-Keung Tang","Fisher Yu"],"abstract":"The recent Segment Anything Model (SAM) represents a big leap in scaling up segmentation models, allowing for powerful zero-shot capabilities and flexible prompting. Despite being trained with 1.1 billion masks, SAM's mask prediction quality falls short in many cases, particularly when dealing with objects that have intricate structures. We propose HQ-SAM, equipping SAM with the ability to accurately segment any object, while maintaining SAM's original promptable design, efficiency, and zero-shot generalizability. Our careful design reuses and preserves the pre-trained model weights of SAM, while only introducing minimal additional parameters and computation. We design a learnable High-Quality Output Token, which is injected into SAM's mask decoder and is responsible for predicting the high-quality mask. Instead of only applying it on mask-decoder features, we first fuse them with early and final ViT features for improved mask details. To train our introduced learnable parameters, we compose a dataset of 44K fine-grained masks from several sources. HQ-SAM is only trained on the introduced detaset of 44k masks, which takes only 4 hours on 8 GPUs. We show the efficacy of HQ-SAM in a suite of 10 diverse segmentation datasets across different downstream tasks, where 8 out of them are evaluated in a zero-shot transfer protocol. Our code and pretrained models are at https://github.com/SysCV/SAM-HQ.","url_abs":"https://arxiv.org/abs/2306.01567v2","url_pdf":"https://arxiv.org/pdf/2306.01567v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"segment-anything-in-high-quality","repo_url":"https://github.com/syscv/sam-hq","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"segment-anything-in-high-quality","repo_url":"https://github.com/sqhuang0103/samreg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"segment-anything-in-high-quality","repo_url":"https://github.com/IDEA-Research/Grounded-Segment-Anything","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"segment-anything-in-high-quality","repo_url":"https://github.com/huggingface/transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"zero-shot-segmentation","task_name":"Zero Shot Segmentation"},{"task_slug":"zero-shot-instance-segmentation","task_name":"Zero-Shot Instance Segmentation"}],"methods":[{"method_slug":"sam","method_name":"SAM"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/zero-shot-segmentation-on-segmentation-in-the","task":"Zero Shot Segmentation","dataset":"Segmentation in the Wild","model":"Grounded HQ-SAM","rank_in_archive_order":1,"of":12,"metrics":{"Mean AP":"49.6"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.01567","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.01567"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/transformers","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/SysCV/SAM-HQ","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IDEA-Research/Grounded-Segment-Anything","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sqhuang0103/samreg","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/syscv/sam-hq","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_violates":1,"ran_honours":1,"ran_fixture":1,"unverified":14},"by_repo_kind":{"official":{"samples":14,"ran":3,"repositories":1},"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"7ceaad42da42edd5","entry":"compute_axial_cis","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"sam-hq2/sam2/modeling/position_encoding.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/sam-hq2/sam2/modeling/position_encoding.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7ceaad42da42edd5"}},{"code_sha256_prefix":"bb060d29525f90eb","entry":"init_t_xy","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"sam-hq2/sam2/modeling/position_encoding.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/sam-hq2/sam2/modeling/position_encoding.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bb060d29525f90eb"}},{"code_sha256_prefix":"f8f09f78b474ee7c","entry":"reshape_for_broadcast","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"sam-hq2/sam2/modeling/position_encoding.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/sam-hq2/sam2/modeling/position_encoding.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f8f09f78b474ee7c"}},{"code_sha256_prefix":"c5057a50b489acc7","entry":"ImageEncoderViT","repo":"sqhuang0103/samreg","repo_kind":"listed","path":"model/segment_anything/modeling/sam.py","file_url":"https://github.com/sqhuang0103/samreg/blob/HEAD/model/segment_anything/modeling/sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5057a50b489acc7"}},{"code_sha256_prefix":"34833c5f5b063e06","entry":"MaskDecoder","repo":"sqhuang0103/samreg","repo_kind":"listed","path":"model/segment_anything/modeling/sam.py","file_url":"https://github.com/sqhuang0103/samreg/blob/HEAD/model/segment_anything/modeling/sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"34833c5f5b063e06"}},{"code_sha256_prefix":"365b6e55c7dfeb2a","entry":"MaskDecoderHQ","repo":"syscv/sam-hq","repo_kind":"official","path":"segment_anything/modeling/mask_decoder_hq.py","file_url":"https://github.com/syscv/sam-hq/blob/HEAD/segment_anything/modeling/mask_decoder_hq.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"365b6e55c7dfeb2a"}},{"code_sha256_prefix":"7ee31020ecfc08df","entry":"Sam","repo":"sqhuang0103/samreg","repo_kind":"listed","path":"model/segment_anything/modeling/sam.py","file_url":"https://github.com/sqhuang0103/samreg/blob/HEAD/model/segment_anything/modeling/sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7ee31020ecfc08df"}},{"code_sha256_prefix":"f9850c1d4acdbb5a","entry":"build_sam2","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"sam-hq2/sam2/build_sam.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/sam-hq2/sam2/build_sam.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9850c1d4acdbb5a"}},{"code_sha256_prefix":"71ae361c498f6c02","entry":"build_sam2_hq_video_predictor","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"sam-hq2/sam2/build_sam.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/sam-hq2/sam2/build_sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"71ae361c498f6c02"}},{"code_sha256_prefix":"c1b9e144954a57e5","entry":"build_sam2_video_predictor","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"sam-hq2/sam2/build_sam.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/sam-hq2/sam2/build_sam.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c1b9e144954a57e5"}},{"code_sha256_prefix":"f9bc5f31ce61cee8","entry":"build_sam_vit_b","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"segment_anything/build_sam_baseline.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/segment_anything/build_sam_baseline.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bc5f31ce61cee8"}},{"code_sha256_prefix":"ce435ccfda4628ae","entry":"build_sam_vit_b","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"segment_anything/build_sam.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/segment_anything/build_sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce435ccfda4628ae"}},{"code_sha256_prefix":"6b77c1f11fff3ed6","entry":"build_sam_vit_h","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"segment_anything/build_sam_baseline.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/segment_anything/build_sam_baseline.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6b77c1f11fff3ed6"}},{"code_sha256_prefix":"28898b68c1790b24","entry":"build_sam_vit_h","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"segment_anything/build_sam.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/segment_anything/build_sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"28898b68c1790b24"}},{"code_sha256_prefix":"3f8890e695469246","entry":"build_sam_vit_l","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"segment_anything/build_sam_baseline.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/segment_anything/build_sam_baseline.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3f8890e695469246"}},{"code_sha256_prefix":"c11d78a854b4aa23","entry":"build_sam_vit_l","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"segment_anything/build_sam.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/segment_anything/build_sam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c11d78a854b4aa23"}},{"code_sha256_prefix":"9d5a5692e595b123","entry":"register_tiny_vit_model","repo":"SysCV/SAM-HQ","repo_kind":"official","path":"segment_anything/modeling/tiny_vit_sam.py","file_url":"https://github.com/SysCV/SAM-HQ/blob/HEAD/segment_anything/modeling/tiny_vit_sam.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d5a5692e595b123"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}