{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contextual-masked-auto-encoder-for-retrieval","title":"Dial-MAE: ConTextual Masked Auto-Encoder for Retrieval-based Dialogue Systems","arxiv_id":"2306.04357","date":"2023-06-07","proceeding":null,"authors":["Zhenpeng Su","Xing Wu","Wei Zhou","Guangyuan Ma","Songlin Hu"],"abstract":"Dialogue response selection aims to select an appropriate response from several candidates based on a given user and system utterance history. Most existing works primarily focus on post-training and fine-tuning tailored for cross-encoders. However, there are no post-training methods tailored for dense encoders in dialogue response selection. We argue that when the current language model, based on dense dialogue systems (such as BERT), is employed as a dense encoder, it separately encodes dialogue context and response, leading to a struggle to achieve the alignment of both representations. Thus, we propose Dial-MAE (Dialogue Contextual Masking Auto-Encoder), a straightforward yet effective post-training technique tailored for dense encoders in dialogue response selection. Dial-MAE uses an asymmetric encoder-decoder architecture to compress the dialogue semantics into dense vectors, which achieves better alignment between the features of the dialogue context and response. Our experiments have demonstrated that Dial-MAE is highly effective, achieving state-of-the-art performance on two commonly evaluated benchmarks.","url_abs":"https://arxiv.org/abs/2306.04357v5","url_pdf":"https://arxiv.org/pdf/2306.04357v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contextual-masked-auto-encoder-for-retrieval","repo_url":"https://github.com/suu990901/Dial-MAE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"conversational-response-selection","task_name":"Conversational Response Selection"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"masked-language-modeling","task_name":"Masked Language Modeling"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/conversational-response-selection-on-e","task":"Conversational Response Selection","dataset":"E-commerce","model":"DialMAE","rank_in_archive_order":2,"of":15,"metrics":{"R10@1":"0.930","R10@2":"0.977","R10@5":"0.997"},"uses_additional_data":false},{"leaderboard":"/sota/conversational-response-selection-on-ubuntu-1","task":"Conversational Response Selection","dataset":"Ubuntu Dialogue (v1, Ranking)","model":"Dial-MAE","rank_in_archive_order":1,"of":25,"metrics":{"R10@1":"0.918","R10@2":"0.964","R10@5":"0.993"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.04357","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}