{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/egoshots-an-ego-vision-life-logging-dataset","title":"Egoshots, an ego-vision life-logging dataset and semantic fidelity metric to evaluate diversity in image captioning models","arxiv_id":"2003.11743","date":"2020-03-26","proceeding":null,"authors":["Pranav Agarwal","Alejandro Betancourt","Vana Panagiotou","Natalia Díaz-Rodríguez"],"abstract":"Image captioning models have been able to generate grammatically correct and human understandable sentences. However most of the captions convey limited information as the model used is trained on datasets that do not caption all possible objects existing in everyday life. Due to this lack of prior information most of the captions are biased to only a few objects present in the scene, hence limiting their usage in daily life. In this paper, we attempt to show the biased nature of the currently existing image captioning models and present a new image captioning dataset, Egoshots, consisting of 978 real life images with no captions. We further exploit the state of the art pre-trained image captioning and object recognition networks to annotate our images and show the limitations of existing works. Furthermore, in order to evaluate the quality of the generated captions, we propose a new image captioning metric, object based Semantic Fidelity (SF). Existing image captioning metrics can evaluate a caption only in the presence of their corresponding annotations; however, SF allows evaluating captions generated for images without annotations, making it highly useful for real life generated captions.","url_abs":"https://arxiv.org/abs/2003.11743v2","url_pdf":"https://arxiv.org/pdf/2003.11743v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"egoshots-an-ego-vision-life-logging-dataset","repo_url":"https://github.com/NataliaDiaz/Egoshots","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-2.0"}},{"paper_slug":"egoshots-an-ego-vision-life-logging-dataset","repo_url":"https://github.com/Pranav21091996/Semantic_Fidelity-and-Egoshots","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"object-recognition","task_name":"Object Recognition"}],"methods":[],"datasets_introduced":[{"slug":"egoshots","name":"EgoShots","full_name":null}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2003.11743","atlas_url":"https://app.syntology.ai/?focus=2003.11743","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2003.11743"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/NataliaDiaz/Egoshots","reach":{"status":"ok","spdx":"GPL-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Pranav21091996/Semantic_Fidelity-and-Egoshots","reach":null}],"summary":{"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"c64fced4e73ca43d","entry":"Semantic_Fidelity","repo":"Pranav21091996/Semantic_Fidelity-and-Egoshots","repo_kind":"official","path":"metrics.py","file_url":"https://github.com/Pranav21091996/Semantic_Fidelity-and-Egoshots/blob/HEAD/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c64fced4e73ca43d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}