{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cocog-controllable-visual-stimuli-generation","title":"CoCoG: Controllable Visual Stimuli Generation based on Human Concept Representations","arxiv_id":"2404.16482","date":"2024-04-25","proceeding":null,"authors":["Chen Wei","Jiachen Zou","Dietmar Heinke","Quanying Liu"],"abstract":"A central question for cognitive science is to understand how humans process visual objects, i.e, to uncover human low-dimensional concept representation space from high-dimensional visual stimuli. Generating visual stimuli with controlling concepts is the key. However, there are currently no generative models in AI to solve this problem. Here, we present the Concept based Controllable Generation (CoCoG) framework. CoCoG consists of two components, a simple yet efficient AI agent for extracting interpretable concept and predicting human decision-making in visual similarity judgment tasks, and a conditional generation model for generating visual stimuli given the concepts. We quantify the performance of CoCoG from two aspects, the human behavior prediction accuracy and the controllable generation ability. The experiments with CoCoG indicate that 1) the reliable concept embeddings in CoCoG allows to predict human behavior with 64.07\\% accuracy in the THINGS-similarity dataset; 2) CoCoG can generate diverse objects through the control of concepts; 3) CoCoG can manipulate human similarity judgment behavior by intervening key concepts. CoCoG offers visual objects with controlling concepts to advance our understanding of causality in human cognition. The code of CoCoG is available at \\url{https://github.com/ncclab-sustech/CoCoG}.","url_abs":"https://arxiv.org/abs/2404.16482v1","url_pdf":"https://arxiv.org/pdf/2404.16482v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cocog-controllable-visual-stimuli-generation","repo_url":"https://github.com/ncclab-sustech/cocog","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"ai-agent","task_name":"AI Agent"},{"task_slug":"decision-making","task_name":"Decision Making"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.16482","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.16482"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ncclab-sustech/CoCoG","reach":null}],"summary":{"ran_violates":1,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"89335701427b909e","entry":"Soft_Min_SNR","repo":"ncclab-sustech/CoCoG","repo_kind":"official","path":"diffusion_prior.py","file_url":"https://github.com/ncclab-sustech/CoCoG/blob/HEAD/diffusion_prior.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"89335701427b909e"}},{"code_sha256_prefix":"4e8be05e4b3fcf0e","entry":"add_noise_with_snr","repo":"ncclab-sustech/CoCoG","repo_kind":"official","path":"diffusion_prior.py","file_url":"https://github.com/ncclab-sustech/CoCoG/blob/HEAD/diffusion_prior.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4e8be05e4b3fcf0e"}},{"code_sha256_prefix":"8d98eb35732aa4fb","entry":"Pipe","repo":"ncclab-sustech/CoCoG","repo_kind":"official","path":"diffusion_prior.py","file_url":"https://github.com/ncclab-sustech/CoCoG/blob/HEAD/diffusion_prior.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d98eb35732aa4fb"}},{"code_sha256_prefix":"45749b455d0b1396","entry":"requires_grad","repo":"ncclab-sustech/CoCoG","repo_kind":"official","path":"diffusion_prior.py","file_url":"https://github.com/ncclab-sustech/CoCoG/blob/HEAD/diffusion_prior.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45749b455d0b1396"}},{"code_sha256_prefix":"27311a4bc856eb7f","entry":"update_ema","repo":"ncclab-sustech/CoCoG","repo_kind":"official","path":"diffusion_prior.py","file_url":"https://github.com/ncclab-sustech/CoCoG/blob/HEAD/diffusion_prior.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27311a4bc856eb7f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}