{"url":"/dataset/egoexolearn","name":"EgoExoLearn","full_name":null,"description_markdown":"**EgoExoLearn** is a fascinating dataset designed to bridge the gap between **egocentric** and **exocentric** views of procedural activities. \r\n\r\n1. **What Is EgoExoLearn?**   EgoExoLearn is a large-scale dataset that emulates how humans learn by observing others. It focuses on the process of **asynchronous demonstration following**. Participants in the dataset record **egocentric videos** as they perform tasks. These videos are guided by **exocentric-view demonstration videos**. In simpler terms, imagine someone watching a demonstration video (from an external perspective) and then replicating the same task while recording their own point-of-view video.\r\n\r\n2. **Dataset Details:**\r\nEgoExoLearn dataset spans **120 hours** and covers scenarios from daily life and specialized laboratories. It contains:\r\n     - **Egocentric videos**: These are recorded by individuals executing tasks.\r\n     - **Demonstration videos**: These show the same tasks from an external viewpoint.\r\n     - **Gaze data**: High-quality gaze information accompanies the videos.\r\n     - **Multimodal annotations**: Detailed annotations provide context and insights.\r\n\r\n3. **Applications and Benchmarks:**\r\nThe EgoExoLearn dataset serves as a playground for modeling the human ability and thus provides a playground to bridge **asynchronous procedural actions** from different viewpoints.\r\nIt enables new benchmarks such as:\r\n     - **Cross-view association**: Linking actions observed from different perspectives.\r\n     - **Cross-view action planning**: Anticipating and planning actions based on both ego and exo views.\r\n     - **Cross-view referenced skill assessment**: Evaluating skills across viewpoints. Using Exo-view demonstrations as guidance for better ego-view skill assessment.\r\n\r\n4. **Why Is It Important?**\r\n    - Understanding how we map others' activities into our own point of view is a fundamental human skill.\r\n    - EgoExoLearn paves the way for creating AI agents capable of **seamlessly learning by observing humans in the real world**.\r\n\r\n(1) EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric .... https://arxiv.org/html/2403.16182v1.\r\n(2) EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric .... https://egoexolearn.github.io/.\r\n(3) EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric .... https://arxiv.org/abs/2403.16182.\r\n(4) EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric .... https://allainews.com/item/egoexolearn-a-dataset-for-bridging-asynchronous-ego-and-exo-centric-view-of-procedural-activities-in-real-world-2024-03-26/.\r\n(5) undefined. https://github.com/OpenGVLab/EgoExoLearn/.","description_withheld":null,"homepage":"https://github.com/OpenGVLab/EgoExoLearn","introduced_date":"2024-03-24","introduced_date_note":null,"introduced_by":{"paper":"/paper/egoexolearn-a-dataset-for-bridging","title":"EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World","first_author":"Yifei HUANG","url":null},"license":null,"modalities":[],"tasks":[{"name":"Action Recognition","url":"/task/action-recognition-in-videos","datasets_with_task":"/datasets/task/action-recognition-in-videos"},{"name":"Video Retrieval","url":"/task/video-retrieval","datasets_with_task":"/datasets/task/video-retrieval"},{"name":"Video Captioning","url":"/task/video-captioning","datasets_with_task":"/datasets/task/video-captioning"},{"name":"Action Segmentation","url":"/task/action-segmentation","datasets_with_task":"/datasets/task/action-segmentation"},{"name":"Action Quality Assessment","url":"/task/action-quality-assessment","datasets_with_task":"/datasets/task/action-quality-assessment"},{"name":"Action Anticipation","url":"/task/action-anticipation","datasets_with_task":"/datasets/task/action-anticipation"},{"name":"Long Term Anticipation","url":"/task/long-term-anticipation","datasets_with_task":"/datasets/task/long-term-anticipation"}],"languages":[],"variants":["EgoExoLearn"],"data_loaders":[],"num_papers_in_archive":12,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/action-anticipation-on-egoexolearn","task":"Action Anticipation","dataset_variant":"EgoExoLearn","rows":2,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"Action anticipation baseline (co-training, with gaze)","paper":"/paper/egoexolearn-a-dataset-for-bridging","metrics":{"Accuracy":"45.45"},"code_links":[{"title":"opengvlab/egoexolearn","url":"https://github.com/opengvlab/egoexolearn"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"},{"leaderboard":"/sota/action-quality-assessment-on-egoexolearn","task":"Action Quality Assessment","dataset_variant":"EgoExoLearn","rows":2,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"RAAN+TL+Gaze","paper":"/paper/egoexolearn-a-dataset-for-bridging","metrics":{"Accuracy":"81.27"},"code_links":[{"title":"opengvlab/egoexolearn","url":"https://github.com/opengvlab/egoexolearn"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"},{"leaderboard":"/sota/video-retrieval-on-egoexolearn","task":"Video Retrieval","dataset_variant":"EgoExoLearn","rows":2,"metrics":["Accuracy"],"first_row_in_archive_order":{"model":"cross-view association baseline (gaze, val)","paper":"/paper/egoexolearn-a-dataset-for-bridging","metrics":{"Accuracy":"48.35"},"code_links":[{"title":"opengvlab/egoexolearn","url":"https://github.com/opengvlab/egoexolearn"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/egoexolearn-a-dataset-for-bridging","title":"EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World","date":"2024-03-24","rows_on_this_dataset":6,"code_links":1,"syntology":{"read_at":"2026-09-24T18:15:14+00:00","samples_harvested":8,"samples_ran":7,"samples_unverified":1,"pointer_only_for_licence":0,"claim":"Per-sample execution on synthesized fixtures; not a correctness claim."}}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":1,"samples_harvested":8,"samples_ran":7,"samples_unverified":1,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}