{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-08630","title":"ExPO-HM: Learning to Explain-then-Detect for Hateful Meme Detection","arxiv_id":"2510.08630","date":"2025-10-08","proceeding":"ICLR","authors":["Jingbiao Mei","Mingsheng Sun","Jinghong Chen","Pengda Qin","Yuhong Li","Da Chen","Bill Byrne"],"abstract":"Hateful memes have emerged as a particularly challenging form of online abuse, motivating the development of automated detection systems. Most prior approaches rely on direct detection, producing only binary predictions. Such models fail to provide the context and explanations that real-world moderation requires. Recent Explain-then-Detect approaches, using Chain-of-Thought prompting or LMM agents, perform worse than simple SFT baselines, and even advanced post-training methods such as GRPO fail to close the gap. Our analysis identifies two key issues of such systems: important policy-relevant cues such as targets and attack types are not hypothesized by the model as a likely explanation; and the binary reward signal is insufficient to guide reasoning. To address these challenges, we propose ExPO-HM (Explain-then-Detect Policy Optimization for Hateful Memes), inspired by the training and evaluation process of human annotators. ExPO-HM combines SFT warmup, GRPO with curriculum learning, and Conditional Decision Entropy (CDE) as both metric and reward for reasoning quality. Across three hateful meme benchmarks, ExPO-HM achieves state-of-the-art performance on binary detection, fine-grained classification, and reasoning quality, with up to 15\\% and 17\\% F1 improvement over the GRPO and DPO baselines, respectively. By moving hateful meme detection from simple binary alarms to explanation-driven detection, ExPO-HM provides accurate, interpretable, and actionable moderation support. Code available at https://github.com/JingbiaoMei/ExPO-HM","url_abs":"https://arxiv.org/abs/2510.08630","url_pdf":"https://arxiv.org/pdf/2510.08630","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2510.08630","atlas_url":"https://app.syntology.ai/?focus=2510.08630","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.08630"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/JingbiaoMei/ExPO-HM","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":9},"by_repo_kind":{"found_in_text":{"samples":11,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8896f4e2bafe8abc","entry":"generate_config_from_args","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"verl/verl/model_merger/base_model_merger.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/verl/verl/model_merger/base_model_merger.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8896f4e2bafe8abc"}},{"code_sha256_prefix":"94209884dcb2308e","entry":"get_dynamic_pipeline_shards","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"verl/verl/model_merger/megatron_model_merger.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/verl/verl/model_merger/megatron_model_merger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"94209884dcb2308e"}},{"code_sha256_prefix":"bfc48666e7213f2d","entry":"evaluate_predictions","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/qwen2_hm_inference_vllm.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/qwen2_hm_inference_vllm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bfc48666e7213f2d"}},{"code_sha256_prefix":"016eaa54a3422c75","entry":"evaluate_predictions_fine_grained","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/qwen2_hm_inference_vllm_fine_grained.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/qwen2_hm_inference_vllm_fine_grained.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"016eaa54a3422c75"}},{"code_sha256_prefix":"75b93b44a8ff1eba","entry":"exact_match_reward","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/qwen2_hm_inference_vllm.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/qwen2_hm_inference_vllm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"75b93b44a8ff1eba"}},{"code_sha256_prefix":"b8245157302f5293","entry":"exact_match_reward_fine_grained","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/qwen2_hm_inference_vllm_fine_grained.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/qwen2_hm_inference_vllm_fine_grained.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b8245157302f5293"}},{"code_sha256_prefix":"fc69b093f6c02f8b","entry":"get_img_ids","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/dataset.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc69b093f6c02f8b"}},{"code_sha256_prefix":"b2166367548f64a4","entry":"get_values_from_gt","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/dataset.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b2166367548f64a4"}},{"code_sha256_prefix":"c2ce31fb16ce7dca","entry":"process_split_vllm","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/qwen2_hm_inference_vllm.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/qwen2_hm_inference_vllm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c2ce31fb16ce7dca"}},{"code_sha256_prefix":"0ac06d1a9e26cd7f","entry":"process_split_vllm_fine_grained","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/qwen2_hm_inference_vllm_fine_grained.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/qwen2_hm_inference_vllm_fine_grained.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ac06d1a9e26cd7f"}},{"code_sha256_prefix":"e37f2704b4234a7e","entry":"to_device","repo":"JingbiaoMei/ExPO-HM","repo_kind":"found_in_text","path":"eval/inference/dataset.py","file_url":"https://github.com/JingbiaoMei/ExPO-HM/blob/HEAD/eval/inference/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e37f2704b4234a7e"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_api"},"syntology_extracted_results":null}