{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/countgd-multi-modal-open-world-counting","title":"CountGD: Multi-Modal Open-World Counting","arxiv_id":"2407.04619","date":"2024-07-05","proceeding":null,"authors":["Niki Amini-Naieni","Tengda Han","Andrew Zisserman"],"abstract":"The goal of this paper is to improve the generality and accuracy of open-vocabulary object counting in images. To improve the generality, we repurpose an open-vocabulary detection foundation model (GroundingDINO) for the counting task, and also extend its capabilities by introducing modules to enable specifying the target object to count by visual exemplars. In turn, these new capabilities - being able to specify the target object by multi-modalites (text and exemplars) - lead to an improvement in counting accuracy. We make three contributions: First, we introduce the first open-world counting model, CountGD, where the prompt can be specified by a text description or visual exemplars or both; Second, we show that the performance of the model significantly improves the state of the art on multiple counting benchmarks - when using text only, CountGD is comparable to or outperforms all previous text-only works, and when using both text and visual exemplars, we outperform all previous models; Third, we carry out a preliminary study into different interactions between the text and visual exemplar prompts, including the cases where they reinforce each other and where one restricts the other. The code and an app to test the model are available at https://www.robots.ox.ac.uk/~vgg/research/countgd/.","url_abs":"https://arxiv.org/abs/2407.04619v1","url_pdf":"https://arxiv.org/pdf/2407.04619v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"countgd-multi-modal-open-world-counting","repo_url":"https://github.com/niki-amini-naieni/CountGD","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"countgd-multi-modal-open-world-counting","repo_url":"https://github.com/niki-amini-naieni/countx","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"object-counting","task_name":"Object Counting"},{"task_slug":"open-vocabulary-object-counting","task_name":"Open-vocabulary object counting"},{"task_slug":"zero-shot-counting","task_name":"Zero-Shot Counting"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/object-counting-on-fsc147","task":"Object Counting","dataset":"FSC147","model":"CountGD","rank_in_archive_order":1,"of":19,"metrics":{"MAE(test)":"5.74","MAE(val)":"7.1","RMSE(test)":"24.09","RMSE(val)":"26.08"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.04619","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.04619"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/niki-amini-naieni/countx","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/niki-amini-naieni/CountGD","reach":null}],"summary":{"ran":4,"unverified":5},"by_repo_kind":{"listed":{"samples":9,"ran":4,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4cb253256cb32632","entry":"CrossAttention","repo":"niki-amini-naieni/countx","repo_kind":"listed","path":"models_counting_network.py","file_url":"https://github.com/niki-amini-naieni/countx/blob/HEAD/models_counting_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb253256cb32632"}},{"code_sha256_prefix":"59c07cf4d49e5e3c","entry":"DropPath","repo":"niki-amini-naieni/countx","repo_kind":"listed","path":"models_counting_network.py","file_url":"https://github.com/niki-amini-naieni/countx/blob/HEAD/models_counting_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59c07cf4d49e5e3c"}},{"code_sha256_prefix":"6f6aa7aeb7d08957","entry":"Mlp","repo":"niki-amini-naieni/countx","repo_kind":"listed","path":"models_counting_network.py","file_url":"https://github.com/niki-amini-naieni/countx/blob/HEAD/models_counting_network.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f6aa7aeb7d08957"}},{"code_sha256_prefix":"9f204c85d112d40e","entry":"PositionalEncodingsFixed","repo":"niki-amini-naieni/CountGD","repo_kind":"listed","path":"models/GroundingDINO/groundingdino.py","file_url":"https://github.com/niki-amini-naieni/CountGD/blob/HEAD/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f204c85d112d40e"}},{"code_sha256_prefix":"7a6e35a004c4b745","entry":"CountingNetwork","repo":"niki-amini-naieni/countx","repo_kind":"listed","path":"models_counting_network.py","file_url":"https://github.com/niki-amini-naieni/countx/blob/HEAD/models_counting_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a6e35a004c4b745"}},{"code_sha256_prefix":"c0ab3335e60b739b","entry":"CrossAttentionBlock","repo":"niki-amini-naieni/countx","repo_kind":"listed","path":"models_counting_network.py","file_url":"https://github.com/niki-amini-naieni/countx/blob/HEAD/models_counting_network.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0ab3335e60b739b"}},{"code_sha256_prefix":"52f493b1857ba9eb","entry":"GroundingDINO","repo":"niki-amini-naieni/CountGD","repo_kind":"listed","path":"models/GroundingDINO/groundingdino.py","file_url":"https://github.com/niki-amini-naieni/CountGD/blob/HEAD/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52f493b1857ba9eb"}},{"code_sha256_prefix":"daf42163c5ae159d","entry":"TransformerEncoder","repo":"niki-amini-naieni/CountGD","repo_kind":"listed","path":"models/GroundingDINO/groundingdino.py","file_url":"https://github.com/niki-amini-naieni/CountGD/blob/HEAD/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"daf42163c5ae159d"}},{"code_sha256_prefix":"57469407352666b4","entry":"TransformerEncoderLayer","repo":"niki-amini-naieni/CountGD","repo_kind":"listed","path":"models/GroundingDINO/groundingdino.py","file_url":"https://github.com/niki-amini-naieni/CountGD/blob/HEAD/models/GroundingDINO/groundingdino.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57469407352666b4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}