{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/end-to-end-deep-reinforcement-learning-based","title":"End-to-end Deep Reinforcement Learning Based Coreference Resolution","arxiv_id":null,"date":"2019-07-01","proceeding":"ACL 2019 7","authors":["Hongliang Fei","Xu Li","Dingcheng Li","Ping Li"],"abstract":"Recent neural network models have significantly advanced the task of coreference resolution. However, current neural coreference models are usually trained with heuristic loss functions that are computed over a sequence of local decisions. In this paper, we introduce an end-to-end reinforcement learning based coreference resolution model to directly optimize coreference evaluation metrics. Specifically, we modify the state-of-the-art higher-order mention ranking approach in Lee et al. (2018) to a reinforced policy gradient model by incorporating the reward associated with a sequence of coreference linking actions. Furthermore, we introduce maximum entropy regularization for adequate exploration to prevent the model from prematurely converging to a bad local optimum. Our proposed model achieves new state-of-the-art performance on the English OntoNotes v5.0 benchmark.","url_abs":"https://aclanthology.org/P19-1064","url_pdf":"https://aclanthology.org/P19-1064.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"coreference-resolution","task_name":"Coreference Resolution"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"coreference-resolution-1","task_name":"coreference-resolution"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/coreference-resolution-on-conll-2012","task":"Coreference Resolution","dataset":"CoNLL 2012","model":"reinforced model + ELMO","rank_in_archive_order":14,"of":18,"metrics":{"Avg F1":"73.8"},"uses_additional_data":false},{"leaderboard":"/sota/coreference-resolution-on-ontonotes","task":"Coreference Resolution","dataset":"OntoNotes","model":"Reinforced + ELMo","rank_in_archive_order":18,"of":26,"metrics":{"F1":"73.8"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}