{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/distrl-an-asynchronous-distributed","title":"DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agents","arxiv_id":"2410.14803","date":"2024-10-18","proceeding":null,"authors":["Taiyi Wang","Zhihao Wu","Jianheng Liu","Jianye Hao","Jun Wang","Kun Shao"],"abstract":"On-device control agents, especially on mobile devices, are responsible for operating mobile devices to fulfill users' requests, enabling seamless and intuitive interactions. Integrating Multimodal Large Language Models (MLLMs) into these agents enhances their ability to understand and execute complex commands, thereby improving user experience. However, fine-tuning MLLMs for on-device control presents significant challenges due to limited data availability and inefficient online training processes. This paper introduces DistRL, a novel framework designed to enhance the efficiency of online RL fine-tuning for mobile device control agents. DistRL employs centralized training and decentralized data acquisition to ensure efficient fine-tuning in the context of dynamic online interactions. Additionally, the framework is backed by our tailor-made RL algorithm, which effectively balances exploration with the prioritized utilization of collected data to ensure stable and robust training. Our experiments show that, on average, DistRL delivers a 3X improvement in training efficiency and enables training data collection 2.4X faster than the leading synchronous multi-machine methods. Notably, after training, DistRL achieves a 20% relative improvement in success rate compared to state-of-the-art methods on general Android tasks from an open benchmark, significantly outperforming existing approaches while maintaining the same training time. These results validate DistRL as a scalable and efficient solution, offering substantial improvements in both training efficiency and agent performance for real-world, in-the-wild device control tasks.","url_abs":"https://arxiv.org/abs/2410.14803v5","url_pdf":"https://arxiv.org/pdf/2410.14803v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distrl-an-asynchronous-distributed","repo_url":"https://github.com/distrl-lab/distrl-open","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.14803","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.14803"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/distrl-lab/distrl-open","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"listed":{"samples":6,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0cfea60f8eda39cf","entry":"dict_mean","repo":"distrl-lab/distrl-open","repo_kind":"listed","path":"distrl/algorithms/digirl/trainer.py","file_url":"https://github.com/distrl-lab/distrl-open/blob/HEAD/distrl/algorithms/digirl/trainer.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0cfea60f8eda39cf"}},{"code_sha256_prefix":"bdaee053ad0cdb9c","entry":"add_trainer_index","repo":"distrl-lab/distrl-open","repo_kind":"listed","path":"distrl/algorithms/worker_collect_loop.py","file_url":"https://github.com/distrl-lab/distrl-open/blob/HEAD/distrl/algorithms/worker_collect_loop.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bdaee053ad0cdb9c"}},{"code_sha256_prefix":"f6f91e38b8af5fde","entry":"aggregate_trajectories","repo":"distrl-lab/distrl-open","repo_kind":"listed","path":"distrl/algorithms/Async_parallel.py","file_url":"https://github.com/distrl-lab/distrl-open/blob/HEAD/distrl/algorithms/Async_parallel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f6f91e38b8af5fde"}},{"code_sha256_prefix":"dd45821d3b9485af","entry":"aggregate_trajectories","repo":"distrl-lab/distrl-open","repo_kind":"listed","path":"distrl/algorithms/Sync_parallel.py","file_url":"https://github.com/distrl-lab/distrl-open/blob/HEAD/distrl/algorithms/Sync_parallel.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dd45821d3b9485af"}},{"code_sha256_prefix":"61c1c9a5525eff94","entry":"framestack","repo":"distrl-lab/distrl-open","repo_kind":"listed","path":"distrl/algorithms/eval_loop.py","file_url":"https://github.com/distrl-lab/distrl-open/blob/HEAD/distrl/algorithms/eval_loop.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"61c1c9a5525eff94"}},{"code_sha256_prefix":"14189638e184926a","entry":"label_trajectories","repo":"distrl-lab/distrl-open","repo_kind":"listed","path":"distrl/algorithms/offpolicy_train_loop.py","file_url":"https://github.com/distrl-lab/distrl-open/blob/HEAD/distrl/algorithms/offpolicy_train_loop.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14189638e184926a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}