{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/semantic-sentence-similarity-size-does-not","title":"Semantic sentence similarity: size does not always matter","arxiv_id":"2106.08648","date":"2021-06-16","proceeding":null,"authors":["Danny Merkx","Stefan L. Frank","Mirjam Ernestus"],"abstract":"This study addresses the question whether visually grounded speech recognition (VGS) models learn to capture sentence semantics without access to any prior linguistic knowledge. We produce synthetic and natural spoken versions of a well known semantic textual similarity database and show that our VGS model produces embeddings that correlate well with human semantic similarity judgements. Our results show that a model trained on a small image-caption database outperforms two models trained on much larger databases, indicating that database size is not all that matters. We also investigate the importance of having multiple captions per image and find that this is indeed helpful even if the total number of images is lower, suggesting that paraphrasing is a valuable learning signal. While the general trend in the field is to create ever larger datasets to train models on, our findings indicate other characteristics of the database can just as important important.","url_abs":"https://arxiv.org/abs/2106.08648v1","url_pdf":"https://arxiv.org/pdf/2106.08648v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"semantic-sentence-similarity-size-does-not","repo_url":"https://github.com/DannyMerkx/speech2image","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"grounded-language-learning","task_name":"Grounded language learning"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"learning-semantic-representations","task_name":"Learning Semantic Representations"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-embeddings","task_name":"Sentence Embeddings"},{"task_slug":"sentence-similarity","task_name":"Sentence Similarity"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"visual-grounding","task_name":"Visual Grounding"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"cyclical-learning-rate-policy","method_name":"Cyclical Learning Rate Policy"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[{"slug":"spokensts","name":"SpokenSTS","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}