{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/associating-neural-word-embeddings-with-deep","title":"Associating Neural Word Embeddings With Deep Image Representations Using Fisher Vectors","arxiv_id":null,"date":"2015-06-01","proceeding":"CVPR 2015 6","authors":["Benjamin Klein","Guy Lev","Gil Sadeh","Lior Wolf"],"abstract":"In recent years, the problem of associating a sentence with an image has gained a lot of attention. This work continues to push the envelope and makes further progress in the performance of image annotation and image search by a sentence tasks. In this work, we are using the Fisher Vector as a sentence representation by pooling the word2vec embedding of each word in the sentence. The Fisher Vector is typically taken as the gradients of the log-likelihood of descriptors, with respect to the parameters of a Gaussian Mixture Model (GMM). In this work we present two other Mixture Models and derive their Expectation-Maximization and Fisher Vector expressions. The first  is a Laplacian Mixture Model (LMM), which is based on the Laplacian distribution. The second Mixture Model presented is a Hybrid Gaussian-Laplacian Mixture Model (HGLMM) which is based on a weighted geometric mean of the Gaussian and Laplacian distribution. Finally, by using the new Fisher Vectors derived from HGLMMs to represent sentences, we achieve state-of-the-art results for both the image annotation and the image search by a sentence tasks on four benchmarks: Pascal1K, Flickr8K, Flickr30K, and COCO.","url_abs":"http://openaccess.thecvf.com/content_cvpr_2015/html/Klein_Associating_Neural_Word_2015_CVPR_paper.html","url_pdf":"http://openaccess.thecvf.com/content_cvpr_2015/papers/Klein_Associating_Neural_Word_2015_CVPR_paper.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-retrieval-on-youcook2","task":"Video Retrieval","dataset":"YouCook2","model":"HGLMM FV CCA","rank_in_archive_order":14,"of":16,"metrics":{"text-to-video Median Rank":"75","text-to-video R@1":"4.6","text-to-video R@10":"21.6","text-to-video R@5":"14.3"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}