{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multilingual-conceptual-coverage-in-text-to","title":"Multilingual Conceptual Coverage in Text-to-Image Models","arxiv_id":"2306.01735","date":"2023-06-02","proceeding":null,"authors":["Michael Saxon","William Yang Wang"],"abstract":"We propose \"Conceptual Coverage Across Languages\" (CoCo-CroLa), a technique for benchmarking the degree to which any generative text-to-image system provides multilingual parity to its training language in terms of tangible nouns. For each model we can assess \"conceptual coverage\" of a given target language relative to a source language by comparing the population of images generated for a series of tangible nouns in the source language to the population of images generated for each noun under translation in the target language. This technique allows us to estimate how well-suited a model is to a target language as well as identify model-specific weaknesses, spurious correlations, and biases without a-priori assumptions. We demonstrate how it can be used to benchmark T2I models in terms of multilinguality, and how despite its simplicity it is a good proxy for impressive generalization.","url_abs":"https://arxiv.org/abs/2306.01735v1","url_pdf":"https://arxiv.org/pdf/2306.01735v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multilingual-conceptual-coverage-in-text-to","repo_url":"https://github.com/michaelsaxon/cococrola","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.01735","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.01735"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/michaelsaxon/cococrola","reach":null}],"summary":{"ran_fixture":1,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"a2c7c3b972bf96ab","entry":"avg_cos_sim","repo":"michaelsaxon/cococrola","repo_kind":"official","path":"cococrola/analyze/evaluate_model.py","file_url":"https://github.com/michaelsaxon/cococrola/blob/HEAD/cococrola/analyze/evaluate_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a2c7c3b972bf96ab"}},{"code_sha256_prefix":"6d2bd05bce9c4422","entry":"open_image_if_exists","repo":"michaelsaxon/cococrola","repo_kind":"official","path":"cococrola/analyze/evaluate_model.py","file_url":"https://github.com/michaelsaxon/cococrola/blob/HEAD/cococrola/analyze/evaluate_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6d2bd05bce9c4422"}},{"code_sha256_prefix":"e982a6c7f059aa2d","entry":"get_image_embeddings","repo":"michaelsaxon/cococrola","repo_kind":"official","path":"cococrola/analyze/evaluate_model.py","file_url":"https://github.com/michaelsaxon/cococrola/blob/HEAD/cococrola/analyze/evaluate_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e982a6c7f059aa2d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}