{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/leveraging-real-talking-faces-via-self","title":"Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection","arxiv_id":"2201.07131","date":"2022-01-18","proceeding":"CVPR 2022 1","authors":["Alexandros Haliassos","Rodrigo Mira","Stavros Petridis","Maja Pantic"],"abstract":"One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common corruptions such as compression. In this paper, we examine whether we can tackle this issue by harnessing videos of real talking faces, which contain rich information on natural facial appearance and behaviour and are readily available in large quantities online. Our method, termed RealForensics, consists of two stages. First, we exploit the natural correspondence between the visual and auditory modalities in real videos to learn, in a self-supervised cross-modal manner, temporally dense video representations that capture factors such as facial movements, expression, and identity. Second, we use these learned representations as targets to be predicted by our forgery detector along with the usual binary forgery classification task; this encourages it to base its real/fake decision on said factors. We show that our method achieves state-of-the-art performance on cross-manipulation generalisation and robustness experiments, and examine the factors that contribute to its performance. Our results suggest that leveraging natural and unlabelled videos is a promising direction for the development of more robust face forgery detectors.","url_abs":"https://arxiv.org/abs/2201.07131v3","url_pdf":"https://arxiv.org/pdf/2201.07131v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"leveraging-real-talking-faces-via-self","repo_url":"https://github.com/ahaliassos/RealForensics","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deepfake-detection","task_name":"DeepFake Detection"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/deepfake-detection-on-fakeavceleb-1","task":"DeepFake Detection","dataset":"FakeAVCeleb","model":"RealForensics","rank_in_archive_order":2,"of":13,"metrics":{"AP":"95.3","ROC AUC":"97.1"},"uses_additional_data":false},{"leaderboard":"/sota/deepfake-detection-on-fakeavceleb-1","task":"DeepFake Detection","dataset":"FakeAVCeleb","model":"AVBYOL","rank_in_archive_order":8,"of":13,"metrics":{"AP":"73.9","ROC AUC":"59.2"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2201.07131","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.07131"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ahaliassos/RealForensics","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7ed17d14c38c0d69","entry":"affine_transform","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"preprocessing/extract_faces.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/preprocessing/extract_faces.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ed17d14c38c0d69"}},{"code_sha256_prefix":"eb5580e7b0ba294d","entry":"compute_std","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"stage1/ssl_learner.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/stage1/ssl_learner.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb5580e7b0ba294d"}},{"code_sha256_prefix":"63d6be75146d60ff","entry":"concat_all_gather","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"stage1/models/utils.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/stage1/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"63d6be75146d60ff"}},{"code_sha256_prefix":"aefe4fd539bc5731","entry":"concat_all_gather_var_len","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"stage1/models/utils.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/stage1/models/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aefe4fd539bc5731"}},{"code_sha256_prefix":"017a01500c93c164","entry":"concat_all_gather_var_len","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"stage2/metrics.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/stage2/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"017a01500c93c164"}},{"code_sha256_prefix":"07bbc166629ba9b2","entry":"crop_patch","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"preprocessing/extract_faces.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/preprocessing/extract_faces.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"07bbc166629ba9b2"}},{"code_sha256_prefix":"96bde5c83493b6bb","entry":"cut_patch","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"preprocessing/extract_faces.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/preprocessing/extract_faces.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96bde5c83493b6bb"}},{"code_sha256_prefix":"c8aafcf0c34b8ce8","entry":"make_mlp_layer","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"stage1/models/mlps.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/stage1/models/mlps.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8aafcf0c34b8ce8"}},{"code_sha256_prefix":"742ea26f422c2b28","entry":"make_mlp_layer_global","repo":"ahaliassos/RealForensics","repo_kind":"official","path":"stage1/models/mlps.py","file_url":"https://github.com/ahaliassos/RealForensics/blob/HEAD/stage1/models/mlps.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"742ea26f422c2b28"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}