{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/can-llms-simulate-l2-english-dialogue-an","title":"Can LLMs Simulate L2-English Dialogue? An Information-Theoretic Analysis of L1-Dependent Biases","arxiv_id":"2502.14507","date":"2025-02-20","proceeding":null,"authors":["Rena Gao","Xuetong Wu","Tatsuki Kuribayashi","Mingrui Ye","Siya Qi","Carsten Roever","Yuanxing Liu","Zheng Yuan","Jey Han Lau"],"abstract":"This study evaluates Large Language Models' (LLMs) ability to simulate non-native-like English use observed in human second language (L2) learners interfered with by their native first language (L1). In dialogue-based interviews, we prompt LLMs to mimic L2 English learners with specific L1s (e.g., Japanese, Thai, Urdu) across seven languages, comparing their outputs to real L2 learner data. Our analysis examines L1-driven linguistic biases, such as reference word usage and avoidance behaviors, using information-theoretic and distributional density measures. Results show that modern LLMs (e.g., Qwen2.5, LLAMA3.3, DeepseekV3, GPT-4o) replicate L1-dependent patterns observed in human L2 data, with distinct influences from various languages (e.g., Japanese, Korean, and Mandarin significantly affect tense agreement, and Urdu influences noun-verb collocations). Our results reveal the potential of LLMs for L2 dialogue generation and evaluation for future educational applications.","url_abs":"https://arxiv.org/abs/2502.14507v1","url_pdf":"https://arxiv.org/pdf/2502.14507v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"can-llms-simulate-l2-english-dialogue-an","repo_url":"https://github.com/RenaGao/LLMPirorknowledge","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"dialogue-generation","task_name":"Dialogue Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2502.14507","atlas_url":"https://app.syntology.ai/?focus=2502.14507","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.14507"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/RenaGao/LLMPirorknowledge","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6340c353203620a9","entry":"format_prompt","repo":"RenaGao/LLMPirorknowledge","repo_kind":"official","path":"lib/annotation.py","file_url":"https://github.com/RenaGao/LLMPirorknowledge/blob/HEAD/lib/annotation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6340c353203620a9"}},{"code_sha256_prefix":"30f808af7f9e8778","entry":"load_config","repo":"RenaGao/LLMPirorknowledge","repo_kind":"official","path":"lib/L2_model_annotation.py","file_url":"https://github.com/RenaGao/LLMPirorknowledge/blob/HEAD/lib/L2_model_annotation.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30f808af7f9e8778"}},{"code_sha256_prefix":"a495c24b35f9d64c","entry":"read_instruction","repo":"RenaGao/LLMPirorknowledge","repo_kind":"official","path":"lib/annotation.py","file_url":"https://github.com/RenaGao/LLMPirorknowledge/blob/HEAD/lib/annotation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a495c24b35f9d64c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}