{"url":"/dataset/longvale","name":"LongVALE","full_name":null,"description_markdown":"Despite impressive advancements in video understanding, most efforts remain limited to coarse-grained or visual-only video tasks. However, real-world videos encompass omni-modal information (vision, audio, and speech) with a series of events forming a cohesive storyline. The lack of multi-modal video data with fine-grained event annotations and the high cost of manual labeling are major obstacles to comprehensive omni-modality video perception. To address this gap, we propose an automatic pipeline consisting of high-quality multi-modal video filtering, semantically coherent omni-modal event boundary detection, and cross-modal correlation-aware event captioning. In this way, we present LongVALE, the first-ever Vision-Audio-Language Event understanding benchmark comprising 105K omni-modal events with precise temporal boundaries and detailed relation-aware captions within 8.4K high-quality long videos. Further, we build a baseline that leverages LongVALE to enable video large language models (LLMs) for omni-modality fine-grained temporal video understanding for the first time. Extensive experiments demonstrate the effectiveness and great potential of LongVALE in advancing comprehensive multi-modal video understanding.","description_withheld":null,"homepage":"https://ttgeng233.github.io/LongVALE/","introduced_date":"2024-11-29","introduced_date_note":null,"introduced_by":{"paper":"/paper/longvale-vision-audio-language-event","title":"LongVALE: Vision-Audio-Language-Event Benchmark Towards Time-Aware Omni-Modal Perception of Long Videos","first_author":"Tiantian Geng","url":null},"license":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"modalities":[{"name":"Videos","url":"/datasets/modality/videos"},{"name":"Texts","url":"/datasets/modality/texts"},{"name":"Audio","url":"/datasets/modality/audio"},{"name":"Speech","url":"/datasets/modality/speech"}],"tasks":[{"name":"Video Captioning","url":"/task/video-captioning","datasets_with_task":"/datasets/task/video-captioning"},{"name":"Dense Video Captioning","url":"/task/dense-video-captioning","datasets_with_task":"/datasets/task/dense-video-captioning"},{"name":"Audio captioning","url":"/task/audio-captioning","datasets_with_task":"/datasets/task/audio-captioning"},{"name":"Natural Language Moment Retrieval","url":"/task/natural-language-moment-retrieval","datasets_with_task":"/datasets/task/natural-language-moment-retrieval"},{"name":"Moment Retrieval","url":"/task/moment-retrieval","datasets_with_task":"/datasets/task/moment-retrieval"},{"name":"Video Grounding","url":"/task/video-grounding","datasets_with_task":"/datasets/task/video-grounding"},{"name":"audio-visual event localization","url":"/task/audio-visual-event-localization","datasets_with_task":"/datasets/task/audio-visual-event-localization"},{"name":"Audio-Visual Video Captioning","url":"/task/audio-visual-video-captioning","datasets_with_task":"/datasets/task/audio-visual-video-captioning"},{"name":"Video Boundary Captioning","url":"/task/video-boundary-captioning","datasets_with_task":"/datasets/task/video-boundary-captioning"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["LongVALE"],"data_loaders":[],"num_papers_in_archive":2,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}