{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/trajectory-volatility-for-out-of-distribution","title":"Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning","arxiv_id":"2405.14039","date":"2024-05-22","proceeding":null,"authors":["Yiming Wang","Pei Zhang","Baosong Yang","Derek F. Wong","Zhuosheng Zhang","Rui Wang"],"abstract":"Real-world data deviating from the independent and identically distributed (i.i.d.) assumption of in-distribution training data poses security threats to deep networks, thus advancing out-of-distribution (OOD) detection algorithms. Detection methods in generative language models (GLMs) mainly focus on uncertainty estimation and embedding distance measurement, with the latter proven to be most effective in traditional linguistic tasks like summarization and translation. However, another complex generative scenario mathematical reasoning poses significant challenges to embedding-based methods due to its high-density feature of output spaces, but this feature causes larger discrepancies in the embedding shift trajectory between different samples in latent spaces. Hence, we propose a trajectory-based method TV score, which uses trajectory volatility for OOD detection in mathematical reasoning. Experiments show that our method outperforms all traditional algorithms on GLMs under mathematical reasoning scenarios and can be extended to more applications with high-density features in output spaces, such as multiple-choice questions.","url_abs":"https://arxiv.org/abs/2405.14039v2","url_pdf":"https://arxiv.org/pdf/2405.14039v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"trajectory-volatility-for-out-of-distribution","repo_url":"https://github.com/alsace08/ood-math-reasoning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"mathematical-reasoning","task_name":"Mathematical Reasoning"},{"task_slug":"multiple-choice","task_name":"Multiple-choice"},{"task_slug":"ood-detection","task_name":"Out of Distribution (OOD) Detection"},{"task_slug":"out-of-distribution-detection","task_name":"Out-of-Distribution Detection"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.14039","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.14039"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Alsace08/OOD-Math-Reasoning","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alsace08/ood-math-reasoning","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e9d2fd2c0f3bbae8","entry":"OODScore","repo":"alsace08/ood-math-reasoning","repo_kind":"official","path":"Computation/score_utils.py","file_url":"https://github.com/alsace08/ood-math-reasoning/blob/HEAD/Computation/score_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9d2fd2c0f3bbae8"}},{"code_sha256_prefix":"2b79dff7fdb873c4","entry":"softmax","repo":"Alsace08/OOD-Math-Reasoning","repo_kind":"official","path":"Computation/score_utils.py","file_url":"https://github.com/Alsace08/OOD-Math-Reasoning/blob/HEAD/Computation/score_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b79dff7fdb873c4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}