{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/look-for-the-change-learning-object-states","title":"Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos","arxiv_id":"2203.11637","date":"2022-03-22","proceeding":"CVPR 2022 1","authors":["Tomáš Souček","Jean-Baptiste Alayrac","Antoine Miech","Ivan Laptev","Josef Sivic"],"abstract":"Human actions often induce changes of object states such as \"cutting an apple\", \"cleaning shoes\" or \"pouring coffee\". In this paper, we seek to temporally localize object states (e.g. \"empty\" and \"full\" cup) together with the corresponding state-modifying actions (\"pouring coffee\") in long uncurated videos with minimal supervision. The contributions of this work are threefold. First, we develop a self-supervised model for jointly learning state-modifying actions together with the corresponding object states from an uncurated set of videos from the Internet. The model is self-supervised by the causal ordering signal, i.e. initial object state $\\rightarrow$ manipulating action $\\rightarrow$ end state. Second, to cope with noisy uncurated training data, our model incorporates a noise adaptive weighting module supervised by a small number of annotated still images, that allows to efficiently filter out irrelevant videos during training. Third, we collect a new dataset with more than 2600 hours of video and 34 thousand changes of object states, and manually annotate a part of this data to validate our approach. Our results demonstrate substantial improvements over prior work in both action and object state-recognition in video.","url_abs":"https://arxiv.org/abs/2203.11637v1","url_pdf":"https://arxiv.org/pdf/2203.11637v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"look-for-the-change-learning-object-states","repo_url":"https://github.com/soCzech/LookForTheChange","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"object","task_name":"Object"}],"methods":[],"datasets_introduced":[{"slug":"changeit","name":"ChangeIt","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2203.11637","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.11637"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/soCzech/LookForTheChange","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9e495ddef3e05fb0","entry":"changeit_collate_fn","repo":"soCzech/LookForTheChange","repo_kind":"official","path":"method/dataset.py","file_url":"https://github.com/soCzech/LookForTheChange/blob/HEAD/method/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e495ddef3e05fb0"}},{"code_sha256_prefix":"6d121fb7b19d716d","entry":"constrained_argmax","repo":"soCzech/LookForTheChange","repo_kind":"official","path":"method/utils.py","file_url":"https://github.com/soCzech/LookForTheChange/blob/HEAD/method/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6d121fb7b19d716d"}},{"code_sha256_prefix":"6bb21eb30df89617","entry":"crop","repo":"soCzech/LookForTheChange","repo_kind":"official","path":"feature_extraction/transforms.py","file_url":"https://github.com/soCzech/LookForTheChange/blob/HEAD/feature_extraction/transforms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6bb21eb30df89617"}},{"code_sha256_prefix":"b5acbcf2da01bba0","entry":"flip","repo":"soCzech/LookForTheChange","repo_kind":"official","path":"feature_extraction/transforms.py","file_url":"https://github.com/soCzech/LookForTheChange/blob/HEAD/feature_extraction/transforms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5acbcf2da01bba0"}},{"code_sha256_prefix":"2922f4a76291822f","entry":"get_cosine_schedule_with_warmup","repo":"soCzech/LookForTheChange","repo_kind":"official","path":"method/utils.py","file_url":"https://github.com/soCzech/LookForTheChange/blob/HEAD/method/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2922f4a76291822f"}},{"code_sha256_prefix":"d4ff7e0631107324","entry":"rotate","repo":"soCzech/LookForTheChange","repo_kind":"official","path":"feature_extraction/transforms.py","file_url":"https://github.com/soCzech/LookForTheChange/blob/HEAD/feature_extraction/transforms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d4ff7e0631107324"}},{"code_sha256_prefix":"50b0cc2d6123e67f","entry":"select_correct_classes","repo":"soCzech/LookForTheChange","repo_kind":"official","path":"method/utils.py","file_url":"https://github.com/soCzech/LookForTheChange/blob/HEAD/method/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"50b0cc2d6123e67f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}