{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prompt-enhanced-network-for-hateful-meme","title":"Prompt-enhanced Network for Hateful Meme Classification","arxiv_id":"2411.07527","date":"2024-11-12","proceeding":null,"authors":["Junxi Liu","Yanyan Feng","Jiehai Chen","Yun Xue","Fenghuan Li"],"abstract":"The dynamic expansion of social media has led to an inundation of hateful memes on media platforms, accentuating the growing need for efficient identification and removal. Acknowledging the constraints of conventional multimodal hateful meme classification, which heavily depends on external knowledge and poses the risk of including irrelevant or redundant content, we developed Pen -- a prompt-enhanced network framework based on the prompt learning approach. Specifically, after constructing the sequence through the prompt method and encoding it with a language model, we performed region information global extraction on the encoded sequence for multi-view perception. By capturing global information about inference instances and demonstrations, Pen facilitates category selection by fully leveraging sequence information. This approach significantly improves model classification accuracy. Additionally, to bolster the model's reasoning capabilities in the feature space, we introduced prompt-aware contrastive learning into the framework to improve the quality of sample feature distributions. Through extensive ablation experiments on two public datasets, we evaluate the effectiveness of the Pen framework, concurrently comparing it with state-of-the-art model baselines. Our research findings highlight that Pen surpasses manual prompt methods, showcasing superior generalization and classification accuracy in hateful meme classification tasks. Our code is available at https://github.com/juszzi/Pen.","url_abs":"https://arxiv.org/abs/2411.07527v1","url_pdf":"https://arxiv.org/pdf/2411.07527v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prompt-enhanced-network-for-hateful-meme","repo_url":"https://github.com/juszzi/pen","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"hateful-meme-classification","task_name":"Hateful Meme Classification"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"meme-classification","task_name":"Meme Classification"},{"task_slug":"prompt-learning","task_name":"Prompt Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.07527","atlas_url":"https://app.syntology.ai/?focus=2411.07527","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.07527"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/juszzi/Pen","reach":null}],"summary":{"ran":4,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"5f1c246ba093aa19","entry":"EncoderLayer","repo":"juszzi/Pen","repo_kind":"official","path":"Pen/rela_encoder.py","file_url":"https://github.com/juszzi/Pen/blob/HEAD/Pen/rela_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f1c246ba093aa19"}},{"code_sha256_prefix":"295c7cd78c0c5615","entry":"MultiHeadedAttention","repo":"juszzi/Pen","repo_kind":"official","path":"Pen/rela_encoder.py","file_url":"https://github.com/juszzi/Pen/blob/HEAD/Pen/rela_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"295c7cd78c0c5615"}},{"code_sha256_prefix":"310aa085b41983d9","entry":"PositionwiseFeedForward","repo":"juszzi/Pen","repo_kind":"official","path":"Pen/rela_encoder.py","file_url":"https://github.com/juszzi/Pen/blob/HEAD/Pen/rela_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"310aa085b41983d9"}},{"code_sha256_prefix":"2f0602f40d97d77f","entry":"Rela_Module","repo":"juszzi/Pen","repo_kind":"official","path":"Pen/rela_encoder.py","file_url":"https://github.com/juszzi/Pen/blob/HEAD/Pen/rela_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2f0602f40d97d77f"}},{"code_sha256_prefix":"a714be7300922d46","entry":"attention","repo":"juszzi/Pen","repo_kind":"official","path":"Pen/rela_encoder.py","file_url":"https://github.com/juszzi/Pen/blob/HEAD/Pen/rela_encoder.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a714be7300922d46"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}