{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/trackdiffusion-multi-object-tracking-data","title":"TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models","arxiv_id":"2312.00651","date":"2023-12-01","proceeding":null,"authors":["Pengxiang Li","Kai Chen","Zhili Liu","Ruiyuan Gao","Lanqing Hong","Guo Zhou","Hua Yao","Dit-yan Yeung","Huchuan Lu","Xu Jia"],"abstract":"Despite remarkable achievements in video synthesis, achieving granular control over complex dynamics, such as nuanced movement among multiple interacting objects, still presents a significant hurdle for dynamic world modeling, compounded by the necessity to manage appearance and disappearance, drastic scale changes, and ensure consistency for instances across frames. These challenges hinder the development of video generation that can faithfully mimic real-world complexity, limiting utility for applications requiring high-level realism and controllability, including advanced scene simulation and training of perception systems. To address that, we propose TrackDiffusion, a novel video generation framework affording fine-grained trajectory-conditioned motion control via diffusion models, which facilitates the precise manipulation of the object trajectories and interactions, overcoming the prevalent limitation of scale and continuity disruptions. A pivotal component of TrackDiffusion is the instance enhancer, which explicitly ensures inter-frame consistency of multiple objects, a critical factor overlooked in the current literature. Moreover, we demonstrate that generated video sequences by our TrackDiffusion can be used as training data for visual perception models. To the best of our knowledge, this is the first work to apply video diffusion models with tracklet conditions and demonstrate that generated frames can be beneficial for improving the performance of object trackers.","url_abs":"https://arxiv.org/abs/2312.00651v2","url_pdf":"https://arxiv.org/pdf/2312.00651v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"trackdiffusion-multi-object-tracking-data","repo_url":"https://github.com/pixeli99/svd_xtend","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"multi-object-tracking","task_name":"Multi-Object Tracking"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-tracking","task_name":"Object Tracking"},{"task_slug":"video-generation","task_name":"Video Generation"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"copy-paste","method_name":"Copy-Paste"},{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.00651","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.00651"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pixeli99/svd_xtend","reach":null}],"summary":{"ran_draft_wrong":1,"ran_fixture":1},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"fface7ec62e39a3b","entry":"rand_log_normal","repo":"pixeli99/svd_xtend","repo_kind":"listed","path":"train_svd.py","file_url":"https://github.com/pixeli99/svd_xtend/blob/HEAD/train_svd.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fface7ec62e39a3b"}},{"code_sha256_prefix":"4db8d7280000a4d9","entry":"tensor_to_vae_latent","repo":"pixeli99/svd_xtend","repo_kind":"listed","path":"train_svd.py","file_url":"https://github.com/pixeli99/svd_xtend/blob/HEAD/train_svd.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4db8d7280000a4d9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}