{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ptvd-a-large-scale-plot-oriented-multimodal","title":"PTVD: A Large-Scale Plot-Oriented Multimodal Dataset Based on Television Dramas","arxiv_id":"2306.14644","date":"2023-06-26","proceeding":null,"authors":["Chen Li","Xutan Peng","Teng Wang","Yixiao Ge","Mengyang Liu","Xuyuan Xu","Yexin Wang","Ying Shan"],"abstract":"Art forms such as movies and television (TV) dramas are reflections of the real world, which have attracted much attention from the multimodal learning community recently. However, existing corpora in this domain share three limitations: (1) annotated in a scene-oriented fashion, they ignore the coherence within plots; (2) their text lacks empathy and seldom mentions situational context; (3) their video clips fail to cover long-form relationship due to short duration. To address these fundamental issues, using 1,106 TV drama episodes and 24,875 informative plot-focused sentences written by professionals, with the help of 449 human annotators, we constructed PTVD, the first plot-oriented multimodal dataset in the TV domain. It is also the first non-English dataset of its kind. Additionally, PTVD contains more than 26 million bullet screen comments (BSCs), powering large-scale pre-training. Next, aiming to open-source a strong baseline for follow-up works, we developed the multimodal algorithm that attacks different cinema/TV modelling problems with a unified architecture. Extensive experiments on three cognitive-inspired tasks yielded a number of novel observations (some of them being quite counter-intuition), further validating the value of PTVD in promoting multimodal research. The dataset and codes are released at \\url{https://ptvd.github.io/}.","url_abs":"https://arxiv.org/abs/2306.14644v1","url_pdf":"https://arxiv.org/pdf/2306.14644v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ptvd-a-large-scale-plot-oriented-multimodal","repo_url":"https://github.com/palchenli/TVD","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"genre-classification","task_name":"Genre classification"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[{"slug":"ptvd","name":"PTVD","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.14644","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}