{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mtle-a-multitask-learning-encoder-of-visual","title":"MTLE: A Multitask Learning Encoder of Visual Feature Representations for Video and Movie Description","arxiv_id":"1809.07257","date":"2018-09-19","proceeding":null,"authors":["Oliver Nina","Washington Garcia","Scott Clouse","Alper Yilmaz"],"abstract":"Learning visual feature representations for video analysis is a daunting task\nthat requires a large amount of training samples and a proper generalization\nframework. Many of the current state of the art methods for video captioning\nand movie description rely on simple encoding mechanisms through recurrent\nneural networks to encode temporal visual information extracted from video\ndata. In this paper, we introduce a novel multitask encoder-decoder framework\nfor automatic semantic description and captioning of video sequences. In\ncontrast to current approaches, our method relies on distinct decoders that\ntrain a visual encoder in a multitask fashion. Our system does not depend\nsolely on multiple labels and allows for a lack of training data working even\nwith datasets where only one single annotation is viable per video. Our method\nshows improved performance over current state of the art methods in several\nmetrics on multi-caption and single-caption datasets. To the best of our\nknowledge, our method is the first method to use a multitask approach for\nencoding video features. Our method demonstrates its robustness on the Large\nScale Movie Description Challenge (LSMDC) 2017 where our method won the movie\ndescription task and its results were ranked among other competitors as the\nmost helpful for the visually impaired.","url_abs":"http://arxiv.org/abs/1809.07257v1","url_pdf":"http://arxiv.org/pdf/1809.07257v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mtle-a-multitask-learning-encoder-of-visual","repo_url":"https://github.com/OSUPCVLab/VideoToTextDNN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"video-captioning","task_name":"Video Captioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}