{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ask-the-right-questions-active-question","title":"Ask the Right Questions: Active Question Reformulation with Reinforcement Learning","arxiv_id":"1705.07830","date":"2017-05-22","proceeding":"ICLR 2018 1","authors":["Christian Buck","Jannis Bulian","Massimiliano Ciaramita","Wojciech Gajewski","Andrea Gesmundo","Neil Houlsby","Wei Wang"],"abstract":"We frame Question Answering (QA) as a Reinforcement Learning task, an\napproach that we call Active Question Answering. We propose an agent that sits\nbetween the user and a black box QA system and learns to reformulate questions\nto elicit the best possible answers. The agent probes the system with,\npotentially many, natural language reformulations of an initial question and\naggregates the returned evidence to yield the best answer. The reformulation\nsystem is trained end-to-end to maximize answer quality using policy gradient.\nWe evaluate on SearchQA, a dataset of complex questions extracted from\nJeopardy!. The agent outperforms a state-of-the-art base model, playing the\nrole of the environment, and other benchmarks. We also analyze the language\nthat the agent has learned while interacting with the question answering\nsystem. We find that successful question reformulations look quite different\nfrom natural language paraphrases. The agent is able to discover non-trivial\nreformulation strategies that resemble classic information retrieval techniques\nsuch as term re-weighting (tf-idf) and stemming.","url_abs":"http://arxiv.org/abs/1705.07830v3","url_pdf":"http://arxiv.org/pdf/1705.07830v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ask-the-right-questions-active-question","repo_url":"https://github.com/BPYap/Question-Generation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"ask-the-right-questions-active-question","repo_url":"https://github.com/google/active-qa","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1705.07830","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}