{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/question-embeddings-based-on-shannon-entropy","title":"Question Embeddings Based on Shannon Entropy: Solving intent classification task in goal-oriented dialogue system","arxiv_id":"1904.00785","date":"2019-03-25","proceeding":null,"authors":["Aleksandr Perevalov","Daniil Kurushin","Rustam Faizrakhmanov","Farida Khabibrakhmanova"],"abstract":"Question-answering systems and voice assistants are becoming major part of\nclient service departments of many organizations, helping them to reduce the\nlabor costs of staff. In many such systems, there is always natural language\nunderstanding module that solves intent classification task. This task is\ncomplicated because of its case-dependency - every subject area has its own\nsemantic kernel. The state of art approaches for intent classification are\ndifferent machine learning and deep learning methods that use text vector\nrepresentations as input. The basic vector representation models such as Bag of\nwords and TF-IDF generate sparse matrixes, which are becoming very big as the\namount of input data grows. Modern methods such as word2vec and FastText use\nneural networks to evaluate word embeddings with fixed dimension size. As we\nare developing a question-answering system for students and enrollees of the\nPerm National Research Polytechnic University, we have faced the problem of\nuser's intent detection. The subject area of our system is very specific, that\nis why there is a lack of training data. This aspect makes intent\nclassification task more challenging for using state of the art deep learning\nmethods. In this paper, we propose an approach of the questions embeddings\nrepresentation based on calculation of Shannon entropy.The goal of the approach\nis to produce low dimensional question vectors as neural approaches do and to\noutperform related methods, described above in condition of small dataset. We\nevaluate and compare our model with existing ones using logistic regression and\ndataset that contains questions asked by students and enrollees. The data is\nlabeled into six classes. Experimental comparison of proposed approach and\nother models revealed that proposed model performed better in the given task.","url_abs":"http://arxiv.org/abs/1904.00785v1","url_pdf":"http://arxiv.org/pdf/1904.00785v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"question-embeddings-based-on-shannon-entropy","repo_url":"https://github.com/Perevalov/intent_classifier","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"intent-classification","task_name":"Intent Classification"},{"task_slug":"intent-detection","task_name":"Intent Detection"},{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"},{"task_slug":"intent-classification-1","task_name":"intent-classification"}],"methods":[{"method_slug":"logistic-regression","method_name":"Logistic Regression"},{"method_slug":"fasttext","method_name":"fastText"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}