{"url":"/dataset/groundcap","name":"GroundCap","full_name":null,"description_markdown":"GroundCap is a novel grounded image captioning dataset derived from MovieNet, containing 52,350 movie frames with detailed grounded captions. The dataset uniquely features an ID-based system that maintains object identity throughout captions, enables tracking of object interactions, and grounds not only objects but also actions and locations in the scene.\r\n\r\n\r\n### Data Instances\r\n\r\nEach sample in the dataset contains:\r\n- An image (movie frame)\r\n- Object detections with:\r\n - Unique object IDs\r\n - Class labels\r\n - Confidence scores\r\n - Bounding box coordinates\r\n- A grounded caption with three types of grounding tags:\r\n - `<gdo>` for grounding objects (e.g., \"the person\", \"a car\")\r\n - `<gda>` for grounding actions (e.g., \"running\", \"sitting\")\r\n - `<gdl>` for grounding locations (e.g., \"on the bridge\", \"in the kitchen\")\r\n\r\n### Data Fields\r\n\r\n- `id`: Unique identifier for each caption\r\n- `image`: The movie frame being captioned\r\n- `detections`: List of detected objects containing:\r\n  - `id`: Object's unique identifier (integer starting at 0 for each class)\r\n  - `label`: Object class label\r\n  - `score`: Detection confidence score\r\n  - `box`: Bounding box coordinates (x, y, w, h)\r\n- `caption`: Grounded caption text with HTML tags\r\n  - `<gdo>` tags ground object references to detections using `{class}-{id}` as attribute (e.g., `<gdo class=\"person\" person-0>the man</gdo>`)\r\n  - `<gda>` tags ground actions to objects using `{class}-{id}` as attribute (e.g., `<gda class=\"run\" person-0>running</gda>`)\r\n  - `<gdl>` tags ground locations to objects using `{class}-{id}` as attribute (e.g., `<gdl class=\"couch\" couch-0>on the couch</gdl>`)\r\n- `human_annotated`: Boolean indicating whether the caption was automatically generated (False) or human-refined (True)\r\n\r\nMultiple objects can be referenced in a single tag. For instance, `<gdo class=\"person\" person-0 person-1>the two people</gdo>` refers to two detected people,\r\nnamely to the detections with IDs 0 and 1 and the class label \"person\".","description_withheld":null,"homepage":"https://huggingface.co/datasets/daniel3303/GroundCap","introduced_date":"2025-02-19","introduced_date_note":null,"introduced_by":{"paper":"/paper/groundcap-a-visually-grounded-image","title":"GroundCap: A Visually Grounded Image Captioning Dataset","first_author":"Daniel A. P. Oliveira","url":null},"license":null,"modalities":[{"name":"Images","url":"/datasets/modality/images"},{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Image Captioning","url":"/task/image-captioning","datasets_with_task":"/datasets/task/image-captioning"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["GroundCap"],"data_loaders":[],"num_papers_in_archive":1,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}