{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-17104","title":"Scientific Logicality Enriched Methodology for LLM Reasoning: A Practice in Physics","arxiv_id":"2605.17104","date":"2026-05-16","proceeding":"ICML","authors":["Zhaoxin Yu","Nan Xu","Kun Chen","Jiahao Zhao","Lei Wang","Wenji Mao"],"abstract":"With the continuous advancement of reasoning abilities in Large Language Models (LLMs), their application to scientific reasoning tasks has gained significant research attention. Current research primarily emphasizes boosting LLMs' performance on scientific QA benchmarks by training on larger, more comprehensive datasets with extended reasoning chains. However, these approaches neglect the essence of the scientific reasoning process -- logicality, which is the rational foundation to ensure the validity of reasoning steps leading to reliable conclusions. In this work, we make the first systematic investigation into the internal logicality underlying LLM scientific reasoning, and develop a scientific logicality-enriched methodology, including a set of assessment criteria and data sampling methods for logicality-guided training, to improve the logical faithfulness as well as task performance. Further, we take physics, characterized by its diverse logical structures and formalisms, as an exemplar discipline to practise the above methodology. For data construction, we extract scientific problems from academic literature and sample a high-quality dataset exhibiting strong logicality. Experiments based on three different backbone LLMs reveal that: 1) the training data we constructed can effectively improve the scientific logicality in LLM reasoning; and 2) the enriched scientific logicality plays a critical role in solving scientific problems. Code is available at \\href{https://github.com/ScienceOne-AI/PhysLogic}{https://github.com/ScienceOne-AI/PhysLogic}.","url_abs":"https://arxiv.org/abs/2605.17104","url_pdf":"https://arxiv.org/pdf/2605.17104","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2605.17104","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.17104"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ScienceOne-AI/PhysLogic","reach":null}],"summary":{"ran_fixture":3,"ran_honours":1,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"97a9eff543fca366","entry":"calculate_f_score","repo":"ScienceOne-AI/PhysLogic","repo_kind":"found_in_text","path":"src/logicality_metrics.py","file_url":"https://github.com/ScienceOne-AI/PhysLogic/blob/HEAD/src/logicality_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"97a9eff543fca366"}},{"code_sha256_prefix":"cc77a4cf9c3c56ab","entry":"calculate_o_score","repo":"ScienceOne-AI/PhysLogic","repo_kind":"found_in_text","path":"src/logicality_metrics.py","file_url":"https://github.com/ScienceOne-AI/PhysLogic/blob/HEAD/src/logicality_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cc77a4cf9c3c56ab"}},{"code_sha256_prefix":"03a626f3f78b40a5","entry":"calculate_p_score","repo":"ScienceOne-AI/PhysLogic","repo_kind":"found_in_text","path":"src/logicality_metrics.py","file_url":"https://github.com/ScienceOne-AI/PhysLogic/blob/HEAD/src/logicality_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03a626f3f78b40a5"}},{"code_sha256_prefix":"89478f2bb9585b39","entry":"compute_logicality_metrics","repo":"ScienceOne-AI/PhysLogic","repo_kind":"found_in_text","path":"src/logicality_metrics.py","file_url":"https://github.com/ScienceOne-AI/PhysLogic/blob/HEAD/src/logicality_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"89478f2bb9585b39"}},{"code_sha256_prefix":"b06941f10c7e4098","entry":"greedy_matching","repo":"ScienceOne-AI/PhysLogic","repo_kind":"found_in_text","path":"src/logicality_metrics.py","file_url":"https://github.com/ScienceOne-AI/PhysLogic/blob/HEAD/src/logicality_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b06941f10c7e4098"}},{"code_sha256_prefix":"82ec6023fe3765ba","entry":"split_into_sentences","repo":"ScienceOne-AI/PhysLogic","repo_kind":"found_in_text","path":"src/logicality_metrics.py","file_url":"https://github.com/ScienceOne-AI/PhysLogic/blob/HEAD/src/logicality_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"82ec6023fe3765ba"}},{"code_sha256_prefix":"6930ede5fc4b5d27","entry":"calculate_similarity_matrix","repo":"ScienceOne-AI/PhysLogic","repo_kind":"found_in_text","path":"src/logicality_metrics.py","file_url":"https://github.com/ScienceOne-AI/PhysLogic/blob/HEAD/src/logicality_metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6930ede5fc4b5d27"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.AI","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}