{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/openr-an-open-source-framework-for-advanced","title":"OpenR: An Open Source Framework for Advanced Reasoning with Large Language Models","arxiv_id":"2410.09671","date":"2024-10-12","proceeding":null,"authors":["Jun Wang","Meng Fang","Ziyu Wan","Muning Wen","Jiachen Zhu","Anjie Liu","Ziqin Gong","Yan Song","Lei Chen","Lionel M. Ni","Linyi Yang","Ying Wen","Weinan Zhang"],"abstract":"In this technical report, we introduce OpenR, an open-source framework designed to integrate key components for enhancing the reasoning capabilities of large language models (LLMs). OpenR unifies data acquisition, reinforcement learning training (both online and offline), and non-autoregressive decoding into a cohesive software platform. Our goal is to establish an open-source platform and community to accelerate the development of LLM reasoning. Inspired by the success of OpenAI's o1 model, which demonstrated improved reasoning abilities through step-by-step reasoning and reinforcement learning, OpenR integrates test-time compute, reinforcement learning, and process supervision to improve reasoning in LLMs. Our work is the first to provide an open-source framework that explores the core techniques of OpenAI's o1 model with reinforcement learning, achieving advanced reasoning capabilities beyond traditional autoregressive methods. We demonstrate the efficacy of OpenR by evaluating it on the MATH dataset, utilising publicly available data and search methods. Our initial experiments confirm substantial gains, with relative improvements in reasoning and performance driven by test-time computation and reinforcement learning through process reward models. The OpenR framework, including code, models, and datasets, is accessible at https://openreasoner.github.io.","url_abs":"https://arxiv.org/abs/2410.09671v1","url_pdf":"https://arxiv.org/pdf/2410.09671v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"openr-an-open-source-framework-for-advanced","repo_url":"https://github.com/openreasoner/openr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"math","task_name":"Math"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.09671","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.09671"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/openreasoner/openr","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":1,"ran":1,"unverified":8},"by_repo_kind":{"listed":{"samples":7,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"7d1ac1a66da5f14c","entry":"list_of_ints","repo":"openreasoner/openr","repo_kind":"listed","path":"config/config_utils.py","file_url":"https://github.com/openreasoner/openr/blob/HEAD/config/config_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7d1ac1a66da5f14c"}},{"code_sha256_prefix":"0bcbc8c8dbe0be78","entry":"str2bool","repo":"openreasoner/openr","repo_kind":"listed","path":"config/config_utils.py","file_url":"https://github.com/openreasoner/openr/blob/HEAD/config/config_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0bcbc8c8dbe0be78"}},{"code_sha256_prefix":"b78ea990bb05f759","entry":"compute_metrics","repo":"openreasoner/openr","repo_kind":"listed","path":"prm/code/finetune_llama.py","file_url":"https://github.com/openreasoner/openr/blob/HEAD/prm/code/finetune_llama.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b78ea990bb05f759"}},{"code_sha256_prefix":"9675ce6b40daf814","entry":"convert_expr","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"9675ce6b40daf814"}},{"code_sha256_prefix":"d3b7a1bd2b1b514a","entry":"convert_relation","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"d3b7a1bd2b1b514a"}},{"code_sha256_prefix":"540c43cae76ae09a","entry":"gather_scalar","repo":"openreasoner/openr","repo_kind":"listed","path":"distributed/utils.py","file_url":"https://github.com/openreasoner/openr/blob/HEAD/distributed/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"540c43cae76ae09a"}},{"code_sha256_prefix":"ffa3288e3e43d6ee","entry":"init_distributed","repo":"openreasoner/openr","repo_kind":"listed","path":"distributed/utils.py","file_url":"https://github.com/openreasoner/openr/blob/HEAD/distributed/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ffa3288e3e43d6ee"}},{"code_sha256_prefix":"699bf6581fe5aa36","entry":"latex2sympy","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"699bf6581fe5aa36"}},{"code_sha256_prefix":"114a6ef651a1a24a","entry":"preprocess_function","repo":"openreasoner/openr","repo_kind":"listed","path":"gen_rm/fine_tuning.py","file_url":"https://github.com/openreasoner/openr/blob/HEAD/gen_rm/fine_tuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"114a6ef651a1a24a"}},{"code_sha256_prefix":"52f2677707a05132","entry":"preprocess_logits_for_metrics","repo":"openreasoner/openr","repo_kind":"listed","path":"prm/code/finetune_math_shep.py","file_url":"https://github.com/openreasoner/openr/blob/HEAD/prm/code/finetune_math_shep.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52f2677707a05132"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}