{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deepfake-text-detection-in-the-wild","title":"MAGE: Machine-generated Text Detection in the Wild","arxiv_id":"2305.13242","date":"2023-05-22","proceeding":null,"authors":["Yafu Li","Qintong Li","Leyang Cui","Wei Bi","Zhilin Wang","Longyue Wang","Linyi Yang","Shuming Shi","Yue Zhang"],"abstract":"Large language models (LLMs) have achieved human-level text generation, emphasizing the need for effective AI-generated text detection to mitigate risks like the spread of fake news and plagiarism. Existing research has been constrained by evaluating detection methods on specific domains or particular language models. In practical scenarios, however, the detector faces texts from various domains or LLMs without knowing their sources. To this end, we build a comprehensive testbed by gathering texts from diverse human writings and texts generated by different LLMs. Empirical results show challenges in distinguishing machine-generated texts from human-authored ones across various scenarios, especially out-of-distribution. These challenges are due to the decreasing linguistic distinctions between the two sources. Despite challenges, the top-performing detector can identify 86.54% out-of-domain texts generated by a new LLM, indicating the feasibility for application scenarios. We release our resources at https://github.com/yafuly/MAGE.","url_abs":"https://arxiv.org/abs/2305.13242v3","url_pdf":"https://arxiv.org/pdf/2305.13242v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deepfake-text-detection-in-the-wild","repo_url":"https://github.com/yafuly/deepfaketextdetect","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"deepfake-text-detection-in-the-wild","repo_url":"https://github.com/yafuly/mage","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"binary-text-classification","task_name":"Binary text classification"},{"task_slug":"face-swapping","task_name":"Face Swapping"},{"task_slug":"story-generation","task_name":"Story Generation"},{"task_slug":"text-detection","task_name":"Text Detection"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[{"slug":"mage","name":"MAGE","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/binary-text-classification-on-mage-arbitrary","task":"Binary text classification","dataset":"MAGE (Arbitrary-domains & Arbitrary-models)","model":"Longformer","rank_in_archive_order":2,"of":2,"metrics":{"Average Recall":"0.9053"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2305.13242","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.13242"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yafuly/mage","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yafuly/deepfaketextdetect","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3d4bb7881ede0f79","entry":"detect","repo":"yafuly/deepfaketextdetect","repo_kind":"official","path":"deployment/utils.py","file_url":"https://github.com/yafuly/deepfaketextdetect/blob/HEAD/deployment/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3d4bb7881ede0f79"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}