{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/attention-based-neural-text-segmentation","title":"Attention-based Neural Text Segmentation","arxiv_id":"1808.09935","date":"2018-08-29","proceeding":null,"authors":["Pinkesh Badjatiya","Litton J Kurisinkel","Manish Gupta","Vasudeva Varma"],"abstract":"Text segmentation plays an important role in various Natural Language\nProcessing (NLP) tasks like summarization, context understanding, document\nindexing and document noise removal. Previous methods for this task require\nmanual feature engineering, huge memory requirements and large execution times.\nTo the best of our knowledge, this paper is the first one to present a novel\nsupervised neural approach for text segmentation. Specifically, we propose an\nattention-based bidirectional LSTM model where sentence embeddings are learned\nusing CNNs and the segments are predicted based on contextual information. This\nmodel can automatically handle variable sized context information. Compared to\nthe existing competitive baselines, the proposed model shows a performance\nimprovement of ~7% in WinDiff score on three benchmark datasets.","url_abs":"http://arxiv.org/abs/1808.09935v1","url_pdf":"http://arxiv.org/pdf/1808.09935v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"attention-based-neural-text-segmentation","repo_url":"https://github.com/pinkeshbadjatiya/neuralTextSegmentation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"feature-engineering","task_name":"Feature Engineering"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-embeddings","task_name":"Sentence Embeddings"},{"task_slug":"text-segmentation","task_name":"Text Segmentation"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.09935","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}