{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-task-learning-for-detecting-and","title":"Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos","arxiv_id":"1906.06876","date":"2019-06-17","proceeding":null,"authors":["Huy H. Nguyen","Fuming Fang","Junichi Yamagishi","Isao Echizen"],"abstract":"Detecting manipulated images and videos is an important topic in digital media forensics. Most detection methods use binary classification to determine the probability of a query being manipulated. Another important topic is locating manipulated regions (i.e., performing segmentation), which are mostly created by three commonly used attacks: removal, copy-move, and splicing. We have designed a convolutional neural network that uses the multi-task learning approach to simultaneously detect manipulated images and videos and locate the manipulated regions for each query. Information gained by performing one task is shared with the other task and thereby enhance the performance of both tasks. A semi-supervised learning approach is used to improve the network's generability. The network includes an encoder and a Y-shaped decoder. Activation of the encoded features is used for the binary classification. The output of one branch of the decoder is used for segmenting the manipulated regions while that of the other branch is used for reconstructing the input, which helps improve overall performance. Experiments using the FaceForensics and FaceForensics++ databases demonstrated the network's effectiveness against facial reenactment attacks and face swapping attacks as well as its ability to deal with the mismatch condition for previously seen attacks. Moreover, fine-tuning using just a small amount of data enables the network to deal with unseen attacks.","url_abs":"https://arxiv.org/abs/1906.06876v1","url_pdf":"https://arxiv.org/pdf/1906.06876v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-task-learning-for-detecting-and","repo_url":"https://github.com/nii-yamagishilab/ClassNSeg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"tasks":[{"task_slug":"binary-classification","task_name":"Binary Classification"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"face-swapping","task_name":"Face Swapping"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1906.06876","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1906.06876"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nii-yamagishilab/ClassNSeg","reach":{"status":"ok","spdx":"BSD-3-Clause"}}],"summary":{"unverified":5},"by_repo_kind":{"listed":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0d13446e83ecbef5","entry":"extract_face","repo":"nii-yamagishilab/ClassNSeg","repo_kind":"listed","path":"create_dataset_Deepfakes.py","file_url":"https://github.com/nii-yamagishilab/ClassNSeg/blob/HEAD/create_dataset_Deepfakes.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"0d13446e83ecbef5"}},{"code_sha256_prefix":"16f66df5e55a742d","entry":"get_bbox","repo":"nii-yamagishilab/ClassNSeg","repo_kind":"listed","path":"create_dataset_Deepfakes.py","file_url":"https://github.com/nii-yamagishilab/ClassNSeg/blob/HEAD/create_dataset_Deepfakes.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"16f66df5e55a742d"}},{"code_sha256_prefix":"2e049bdf24271e9a","entry":"get_bbox","repo":"nii-yamagishilab/ClassNSeg","repo_kind":"listed","path":"create_dataset_Face2Face.py","file_url":"https://github.com/nii-yamagishilab/ClassNSeg/blob/HEAD/create_dataset_Face2Face.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"2e049bdf24271e9a"}},{"code_sha256_prefix":"2dd5fa80f79d7287","entry":"to_bw","repo":"nii-yamagishilab/ClassNSeg","repo_kind":"listed","path":"create_dataset_Deepfakes.py","file_url":"https://github.com/nii-yamagishilab/ClassNSeg/blob/HEAD/create_dataset_Deepfakes.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"2dd5fa80f79d7287"}},{"code_sha256_prefix":"febed37be8e8b5e5","entry":"to_bw","repo":"nii-yamagishilab/ClassNSeg","repo_kind":"listed","path":"create_dataset_Face2Face.py","file_url":"https://github.com/nii-yamagishilab/ClassNSeg/blob/HEAD/create_dataset_Face2Face.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"febed37be8e8b5e5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}