{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-moments-in-time-learning-and","title":"Multi-Moments in Time: Learning and Interpreting Models for Multi-Action Video Understanding","arxiv_id":"1911.00232","date":"2019-11-01","proceeding":null,"authors":["Mathew Monfort","Bowen Pan","Kandan Ramakrishnan","Alex Andonian","Barry A McNamara","Alex Lascelles","Quanfu Fan","Dan Gutfreund","Rogerio Feris","Aude Oliva"],"abstract":"Videos capture events that typically contain multiple sequential, and simultaneous, actions even in the span of only a few seconds. However, most large-scale datasets built to train models for action recognition in video only provide a single label per video. Consequently, models can be incorrectly penalized for classifying actions that exist in the videos but are not explicitly labeled and do not learn the full spectrum of information present in each video in training. Towards this goal, we present the Multi-Moments in Time dataset (M-MiT) which includes over two million action labels for over one million three second videos. This multi-label dataset introduces novel challenges on how to train and analyze models for multi-action detection. Here, we present baseline results for multi-action recognition using loss functions adapted for long tail multi-label learning, provide improved methods for visualizing and interpreting models trained for multi-label action detection and show the strength of transferring models trained on M-MiT to smaller datasets.","url_abs":"https://arxiv.org/abs/1911.00232v4","url_pdf":"https://arxiv.org/pdf/1911.00232v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-moments-in-time-learning-and","repo_url":"https://github.com/metalbubble/moments_models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-2-Clause"}},{"paper_slug":"multi-moments-in-time-learning-and","repo_url":"https://github.com/zhoubolei/moments_models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"tasks":[{"task_slug":"action-detection","task_name":"Action Detection"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"multi-label-learning","task_name":"Multi-Label Learning"},{"task_slug":"video-understanding","task_name":"Video Understanding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1911.00232","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1911.00232"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/metalbubble/moments_models","reach":{"status":"ok","spdx":"BSD-2-Clause"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhoubolei/moments_models","reach":{"status":"ok","spdx":"BSD-2-Clause"}}],"summary":{"unverified":10},"by_repo_kind":{"listed":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"3972ce0a29ce2650","entry":"conv3x3x3","repo":"metalbubble/moments_models","repo_kind":"listed","path":"models.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"3972ce0a29ce2650"}},{"code_sha256_prefix":"8ad16556fa2664bf","entry":"downsample_basic_block","repo":"metalbubble/moments_models","repo_kind":"listed","path":"models.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"8ad16556fa2664bf"}},{"code_sha256_prefix":"cec5115b0d865d23","entry":"extract_frames","repo":"metalbubble/moments_models","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"cec5115b0d865d23"}},{"code_sha256_prefix":"0aab7fdbbb8ba1be","entry":"generate_image_feats","repo":"metalbubble/moments_models","repo_kind":"listed","path":"compute_prob_dynImg.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/compute_prob_dynImg.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"0aab7fdbbb8ba1be"}},{"code_sha256_prefix":"b5d1496941a59c08","entry":"load_frames","repo":"metalbubble/moments_models","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"b5d1496941a59c08"}},{"code_sha256_prefix":"64e9dbbe51b2dcca","entry":"lsep","repo":"metalbubble/moments_models","repo_kind":"listed","path":"loss_functions.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/loss_functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"64e9dbbe51b2dcca"}},{"code_sha256_prefix":"91fb31d4ddd1b926","entry":"modify_resnets","repo":"metalbubble/moments_models","repo_kind":"listed","path":"models.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"91fb31d4ddd1b926"}},{"code_sha256_prefix":"cc3b5d5adbf6023a","entry":"render_frames","repo":"metalbubble/moments_models","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"cc3b5d5adbf6023a"}},{"code_sha256_prefix":"f4ffd1d5aee4e9b4","entry":"warp","repo":"metalbubble/moments_models","repo_kind":"listed","path":"loss_functions.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/loss_functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"f4ffd1d5aee4e9b4"}},{"code_sha256_prefix":"4a807b90ff83acbf","entry":"wlsep","repo":"metalbubble/moments_models","repo_kind":"listed","path":"loss_functions.py","file_url":"https://github.com/metalbubble/moments_models/blob/HEAD/loss_functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-2-Clause","inline_ok":false,"mcp_get_code":{"code_sha256":"4a807b90ff83acbf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}