{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2604-21045","title":"Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech","arxiv_id":"2604.21045","date":"2026-04-22","proceeding":null,"authors":["Siqi Ouyang","Shuoyang Ding","Oleksii Hrinchuk","Vitaly Lavrukhin","Brian Yan","Boris Ginsburg","Lei Li"],"abstract":"Simultaneous speech translation (SST) generates translations while receiving partial speech input. Recent advances show that large language models (LLMs) can substantially improve SST quality, but at the cost of high computational overhead. To reduce this cost, prior work reformulates SST as a multi-turn dialogue task, enabling full reuse of the LLM's key-value (KV) cache and eliminating redundant feature recomputation. However, this approach relies on supervised fine-tuning (SFT) data in dialogue form, for which few human annotations exist, and existing synthesis methods cannot guarantee data quality. In this work, we propose a Hierarchical Policy Optimization (HPO) approach that post-train models trained on imperfect SFT data. We introduce a hierarchical reward that balances translation quality and latency objectives. Experiments on English to Chinese/German/Japanese demonstrate improvements of over +7 COMET score and +1.25 MetricX score at a latency of 1.5 seconds. Comprehensive ablation studies further validate the effectiveness of different quality rewards, hierarchical reward formulations, and segmentation strategies. Code can be found here https://github.com/owaski/HPO","url_abs":"https://arxiv.org/abs/2604.21045","url_pdf":"https://arxiv.org/pdf/2604.21045","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2604.21045","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2604.21045"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/owaski/HPO","reach":null}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"found_in_text":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8dfcc83eb0213fdf","entry":"calculate_baseline_and_std_per_prompt","repo":"owaski/HPO","repo_kind":"found_in_text","path":"nemo_rl/algorithms/utils.py","file_url":"https://github.com/owaski/HPO/blob/HEAD/nemo_rl/algorithms/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8dfcc83eb0213fdf"}},{"code_sha256_prefix":"4f243e6c96b9cdcd","entry":"calculate_kl_penalty_joschu2020","repo":"owaski/HPO","repo_kind":"found_in_text","path":"nemo_rl/algorithms/utils.py","file_url":"https://github.com/owaski/HPO/blob/HEAD/nemo_rl/algorithms/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f243e6c96b9cdcd"}},{"code_sha256_prefix":"f6ba6d0d9acbfec0","entry":"surpress_user_warnings","repo":"owaski/HPO","repo_kind":"found_in_text","path":"nemo_rl/algorithms/utils.py","file_url":"https://github.com/owaski/HPO/blob/HEAD/nemo_rl/algorithms/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f6ba6d0d9acbfec0"}},{"code_sha256_prefix":"e717663de0f3a400","entry":"from_parallel_logits_to_logprobs","repo":"owaski/HPO","repo_kind":"found_in_text","path":"nemo_rl/distributed/model_utils.py","file_url":"https://github.com/owaski/HPO/blob/HEAD/nemo_rl/distributed/model_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e717663de0f3a400"}},{"code_sha256_prefix":"de770a167b653be2","entry":"gather_jagged_object_lists","repo":"owaski/HPO","repo_kind":"found_in_text","path":"nemo_rl/distributed/collectives.py","file_url":"https://github.com/owaski/HPO/blob/HEAD/nemo_rl/distributed/collectives.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de770a167b653be2"}},{"code_sha256_prefix":"04274251107cee20","entry":"rebalance_nd_tensor","repo":"owaski/HPO","repo_kind":"found_in_text","path":"nemo_rl/distributed/collectives.py","file_url":"https://github.com/owaski/HPO/blob/HEAD/nemo_rl/distributed/collectives.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"04274251107cee20"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}