{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/artemis-affective-language-for-visual-art","title":"ArtEmis: Affective Language for Visual Art","arxiv_id":"2101.07396","date":"2021-01-19","proceeding":"CVPR 2021 1","authors":["Panos Achlioptas","Maks Ovsjanikov","Kilichbek Haydarov","Mohamed Elhoseiny","Leonidas Guibas"],"abstract":"We present a novel large-scale dataset and accompanying machine learning models aimed at providing a detailed understanding of the interplay between visual content, its emotional effect, and explanations for the latter in language. In contrast to most existing annotation datasets in computer vision, we focus on the affective experience triggered by visual artworks and ask the annotators to indicate the dominant emotion they feel for a given image and, crucially, to also provide a grounded verbal explanation for their emotion choice. As we demonstrate below, this leads to a rich set of signals for both the objective content and the affective impact of an image, creating associations with abstract concepts (e.g., \"freedom\" or \"love\"), or references that go beyond what is directly visible, including visual similes and metaphors, or subjective references to personal experiences. We focus on visual art (e.g., paintings, artistic photographs) as it is a prime example of imagery created to elicit emotional responses from its viewers. Our dataset, termed ArtEmis, contains 439K emotion attributions and explanations from humans, on 81K artworks from WikiArt. Building on this data, we train and demonstrate a series of captioning systems capable of expressing and explaining emotions from visual stimuli. Remarkably, the captions produced by these systems often succeed in reflecting the semantic and abstract content of the image, going well beyond systems trained on existing datasets. The collected dataset and developed methods are available at https://artemisdataset.org.","url_abs":"https://arxiv.org/abs/2101.07396v1","url_pdf":"https://arxiv.org/pdf/2101.07396v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"artemis-affective-language-for-visual-art","repo_url":"https://github.com/optas/artemis","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"artemis-affective-language-for-visual-art","repo_url":"https://github.com/Surojit-KB/ARTY","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"artemis-affective-language-for-visual-art","repo_url":"https://github.com/betterzh/sevlm-code","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"artemis-affective-language-for-visual-art","repo_url":"https://github.com/betterzh/sevlm-for-visual-art-comprehension","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"artemis-affective-language-for-visual-art","repo_url":"https://github.com/sendeniz/creative-adverserial-net","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"}],"methods":[],"datasets_introduced":[{"slug":"artemis","name":"ArtEmis","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2101.07396","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2101.07396"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/betterzh/sevlm-for-visual-art-comprehension","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/betterzh/sevlm-code","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/optas/artemis","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sendeniz/creative-adverserial-net","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Surojit-KB/ARTY","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":4},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"5e2978db41658c36","entry":"group_gt_annotations","repo":"optas/artemis","repo_kind":"official","path":"artemis/scripts/preprocess_artemis_data.py","file_url":"https://github.com/optas/artemis/blob/HEAD/artemis/scripts/preprocess_artemis_data.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5e2978db41658c36"}},{"code_sha256_prefix":"a9ad469e43133558","entry":"prep_ans","repo":"betterzh/sevlm-code","repo_kind":"listed","path":"utils/data_utils.py","file_url":"https://github.com/betterzh/sevlm-code/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a9ad469e43133558"}},{"code_sha256_prefix":"1f4e29c084d8fb7a","entry":"process_digit_article","repo":"betterzh/sevlm-code","repo_kind":"listed","path":"utils/data_utils.py","file_url":"https://github.com/betterzh/sevlm-code/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1f4e29c084d8fb7a"}},{"code_sha256_prefix":"ad323281cc450ace","entry":"process_punctuation","repo":"betterzh/sevlm-code","repo_kind":"listed","path":"utils/data_utils.py","file_url":"https://github.com/betterzh/sevlm-code/blob/HEAD/utils/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad323281cc450ace"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}