{"url":"/dataset/a2d-sentences","name":"A2D Sentences","full_name":"Sentences for the Actor-Action Dataset (A2D)","description_markdown":"The Actor-Action Dataset (A2D) by Xu et al. [29] serves as the largest video dataset for the general actor and action segmentation task. It contains 3,782 videos from YouTube with pixel-level labeled actors and their actions. The dataset includes eight different actions, while a total of seven actor classes are considered to perform those actions. We follow [29], who split the dataset into 3,036 training videos and 746 testing videos. \r\n\r\n\r\nAs we are interested in pixel-level actor and action segmentation from sentences, we augment the videos in A2D with natural language descriptions about what each actor is doing in the videos.  Following the guidelines set forth\r\nin [12], we ask our annotators for a discriminative referring expression of each actor instance if multiple objects are considered in a video. The annotation process resulted in a total of 6,656 sentences, including 811 different nouns, 225 verbs and 189 adjectives. Our sentences enrich the actor and action pairs from the A2D dataset with finer granularities. For example, the actor adult in A2D may be annotated with man, woman, person and player in our sentences, while action rolling may also refer to flipping, sliding, moving and running when describing different actors in different scenarios. Our sentences contain on average more words than the ReferIt dataset [12] (7.3 vs 4.7), even when we leave out prepositions, articles and linking verbs (4.5 vs 3.6). This makes sense as our sentences contain a variety of verbs while existing referring expression datasets mostly ignore verbs.","description_withheld":null,"homepage":"https://kgavrilyuk.github.io/publication/actor_action/","introduced_date":"2018-03-20","introduced_date_note":null,"introduced_by":{"paper":"/paper/actor-and-action-video-segmentation-from-a","title":"Actor and Action Video Segmentation from a Sentence","first_author":"Kirill Gavrilyuk","url":null},"license":null,"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Referring Expression Segmentation","url":"/task/referring-expression-segmentation","datasets_with_task":"/datasets/task/referring-expression-segmentation"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["A2D Sentences"],"data_loaders":[{"repo":"https://github.com/adventxaxa/datasetDownload","url":"https://github.com/adventxaxa/datasetDownload","frameworks":[]}],"num_papers_in_archive":31,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/referring-expression-segmentation-on-a2d","task":"Referring Expression Segmentation","dataset_variant":"A2D Sentences","rows":27,"metrics":["AP","IoU overall","IoU mean","Precision@0.5","Precision@0.6","Precision@0.7","Precision@0.8","Precision@0.9"],"first_row_in_archive_order":{"model":"SgMg (Video-Swin-B)","paper":"/paper/spectrum-guided-multi-granularity-referring","metrics":{"AP":"0.585","IoU mean":"0.720","IoU overall":"0.799","Precision@0.5":"0.843","Precision@0.6":"0.822","Precision@0.7":"0.767","Precision@0.8":"0.617","Precision@0.9":"0.259"},"code_links":[{"title":"bo-miao/sgmg","url":"https://github.com/bo-miao/sgmg"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/spectrum-guided-multi-granularity-referring","title":"Spectrum-guided Multi-granularity Referring Video Object Segmentation","date":"2023-07-25","rows_on_this_dataset":1,"code_links":1,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":9,"samples_ran":6,"samples_unverified":3,"pointer_only_for_licence":9,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}},{"paper":"/paper/soc-semantic-assisted-object-cluster-for","title":"SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation","date":"2023-05-26","rows_on_this_dataset":2,"code_links":1,"syntology":null},{"paper":"/paper/multi-attention-network-for-compressed-video","title":"Multi-Attention Network for Compressed Video Referring Object Segmentation","date":"2022-07-26","rows_on_this_dataset":1,"code_links":1,"syntology":null},{"paper":"/paper/modeling-motion-with-multi-modal-features-for","title":"Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation","date":"2022-04-06","rows_on_this_dataset":1,"code_links":1,"syntology":null},{"paper":"/paper/deeply-interleaved-two-stream-encoder-for","title":"Deeply Interleaved Two-Stream Encoder for Referring Video Segmentation","date":"2022-03-30","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/local-global-context-aware-transformer-for","title":"Local-Global Context Aware Transformer for Language-Guided Video Segmentation","date":"2022-03-18","rows_on_this_dataset":1,"code_links":1,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":1,"samples_ran":1,"samples_unverified":0,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}},{"paper":"/paper/language-as-queries-for-referring-video","title":"Language as Queries for Referring Video Object Segmentation","date":"2022-01-03","rows_on_this_dataset":1,"code_links":1,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":8,"samples_ran":7,"samples_unverified":1,"pointer_only_for_licence":8,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}},{"paper":"/paper/end-to-end-referring-video-object","title":"End-to-End Referring Video Object Segmentation with Multimodal Transformers","date":"2021-11-29","rows_on_this_dataset":2,"code_links":2,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":11,"samples_ran":6,"samples_unverified":5,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}},{"paper":"/paper/hierarchical-interaction-network-for-video","title":"Hierarchical interaction network for video object segmentation from referring expressions","date":"2021-11-22","rows_on_this_dataset":2,"code_links":0,"syntology":null},{"paper":"/paper/cross-modal-progressive-comprehension-for","title":"Cross-Modal Progressive Comprehension for Referring Segmentation","date":"2021-05-15","rows_on_this_dataset":2,"code_links":1,"syntology":null},{"paper":"/paper/collaborative-spatial-temporal-modeling-for","title":"Collaborative Spatial-Temporal Modeling for Language-Queried Video Actor Segmentation","date":"2021-05-14","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/clawcranenet-leveraging-object-level-relation","title":"ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentation","date":"2021-03-19","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/referring-segmentation-in-images-and-videos","title":"Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Network","date":"2021-02-09","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/actor-and-action-modular-network-for-text","title":"Actor and Action Modular Network for Text-based Video Segmentation","date":"2020-11-02","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/refvos-a-closer-look-at-referring-expressions","title":"RefVOS: A Closer Look at Referring Expressions for Video Object Segmentation","date":"2020-10-01","rows_on_this_dataset":1,"code_links":2,"syntology":null},{"paper":"/paper/polar-relative-positional-encoding-for-video","title":"Polar Relative Positional Encoding for Video-Language Segmentation","date":"2020-07-20","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/visual-textual-capsule-routing-for-text-based","title":"Visual-Textual Capsule Routing for Text-Based Video Segmentation","date":"2020-06-01","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/context-modulated-dynamic-networks-for-actor","title":"Context Modulated Dynamic Networks for Actor and Action Video Segmentation with Language Queries","date":"2020-04-03","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/asymmetric-cross-guided-attention-network-for","title":"Asymmetric Cross-Guided Attention Network for Actor and Action Video Segmentation From Natural Language Query","date":"2019-10-01","rows_on_this_dataset":1,"code_links":1,"syntology":null},{"paper":"/paper/actor-and-action-video-segmentation-from-a","title":"Actor and Action Video Segmentation from a Sentence","date":"2018-03-20","rows_on_this_dataset":2,"code_links":1,"syntology":null},{"paper":"/paper/tracking-by-natural-language-specification","title":"Tracking by Natural Language Specification","date":"2017-07-01","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/segmentation-from-natural-language","title":"Segmentation from Natural Language Expressions","date":"2016-03-20","rows_on_this_dataset":1,"code_links":4,"syntology":null}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":4,"samples_harvested":29,"samples_ran":20,"samples_unverified":9,"pointer_only_for_licence":17,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}