{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/momentdiff-generative-video-moment-retrieval","title":"MomentDiff: Generative Video Moment Retrieval from Random to Real","arxiv_id":"2307.02869","date":"2023-07-06","proceeding":"NeurIPS 2023 11","authors":["Pandeng Li","Chen-Wei Xie","Hongtao Xie","Liming Zhao","Lei Zhang","Yun Zheng","Deli Zhao","Yongdong Zhang"],"abstract":"Video moment retrieval pursues an efficient and generalized solution to identify the specific temporal segments within an untrimmed video that correspond to a given language description. To achieve this goal, we provide a generative diffusion-based framework called MomentDiff, which simulates a typical human retrieval process from random browsing to gradual localization. Specifically, we first diffuse the real span to random noise, and learn to denoise the random noise to the original span with the guidance of similarity between text and video. This allows the model to learn a mapping from arbitrary random locations to real moments, enabling the ability to locate segments from random initialization. Once trained, MomentDiff could sample random temporal segments as initial guesses and iteratively refine them to generate an accurate temporal boundary. Different from discriminative works (e.g., based on learnable proposals or queries), MomentDiff with random initialized spans could resist the temporal location biases from datasets. To evaluate the influence of the temporal location biases, we propose two anti-bias datasets with location distribution shifts, named Charades-STA-Len and Charades-STA-Mom. The experimental results demonstrate that our efficient framework consistently outperforms state-of-the-art methods on three public benchmarks, and exhibits better generalization and robustness on the proposed anti-bias datasets. The code, model, and anti-bias evaluation datasets are available at https://github.com/IMCCretrieval/MomentDiff.","url_abs":"https://arxiv.org/abs/2307.02869v2","url_pdf":"https://arxiv.org/pdf/2307.02869v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"momentdiff-generative-video-moment-retrieval","repo_url":"https://github.com/imccretrieval/momentdiff","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"moment-retrieval","task_name":"Moment Retrieval"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2307.02869","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.02869"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/imccretrieval/momentdiff","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":4,"ran_draft_wrong":1,"ran_honours":1,"ran_violates":1,"unverified":4},"by_repo_kind":{"official":{"samples":11,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"d6ca656c20df137a","entry":"build_position_encoding","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/position_encoding.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/position_encoding.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d6ca656c20df137a"}},{"code_sha256_prefix":"42c73359ca76f61d","entry":"get_list","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/matcher.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/matcher.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"42c73359ca76f61d"}},{"code_sha256_prefix":"6897880b6ce14c94","entry":"inverse_sigmoid","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/model.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/model.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6897880b6ce14c94"}},{"code_sha256_prefix":"1c17153fb5488283","entry":"mask_logits","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/text_encoder.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/text_encoder.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c17153fb5488283"}},{"code_sha256_prefix":"fbfeec7963c6a061","entry":"prepare_batch_inputs","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/start_end_dataset.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/start_end_dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fbfeec7963c6a061"}},{"code_sha256_prefix":"1c76f7707e1343b4","entry":"prepare_batch_inputs_audio","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/start_end_dataset_audio.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/start_end_dataset_audio.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c76f7707e1343b4"}},{"code_sha256_prefix":"b7639c2e8b83837b","entry":"span_cxw_to_xx","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/span_utils.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/span_utils.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b7639c2e8b83837b"}},{"code_sha256_prefix":"081c8abd57d082fd","entry":"accuracy","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/misc.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"081c8abd57d082fd"}},{"code_sha256_prefix":"a82091666b9db8cf","entry":"multi_head_attention_forward","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/attention.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/attention.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a82091666b9db8cf"}},{"code_sha256_prefix":"7114f025044af9b4","entry":"span_xx_to_cxw","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/span_utils.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/span_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7114f025044af9b4"}},{"code_sha256_prefix":"e4ae5582a2533cdb","entry":"temporal_iou","repo":"imccretrieval/momentdiff","repo_kind":"official","path":"momentdiff/span_utils.py","file_url":"https://github.com/imccretrieval/momentdiff/blob/HEAD/momentdiff/span_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e4ae5582a2533cdb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}