{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusionvid-denoising-object-boxes-with","title":"DiffusionVID: Denoising Object Boxes with Spatio-temporal Conditioning for Video Object Detection","arxiv_id":null,"date":"2023-10-30","proceeding":"IEEE Access 2023 10","authors":["Si-Dong Roh","Ki-Seok Chung"],"abstract":"Several existing still image object detectors suffer from image deterioration in videos, such as motion blur, camera defocus, and partial occlusion. We present DiffusionVID, a diffusion model-based video object detector, that exploits spatio-temporal conditioning. Inspired by the diffusion model, DiffusionVID refines random noise boxes to obtain the original object boxes in a video sequence. To effectively refine the box from the degraded images in the videos, we used three novel approaches: cascade refinement, dynamic core-set conditioning, and local batch refinement. The cascade refinement architecture effectively collects information from object regions, whereas the dynamic core-set conditioning further improves the denoising quality using adaptive conditional guidance based on the spatio-temporal core-set. Local batch refinement significantly improves the refinement speed by exploiting GPU parallelism. On the standard and widely used ImageNet-VID benchmark, our DiffusionVID with the ResNet-101 and Swin-Base backbones achieves 86.9 mAP @ 46.6 FPS and 92.4 mAP @ 27.0 FPS, respectively, which is state-of-the-art performance. To the best of the authors’ knowledge, this is the first video object detector based on a diffusion model. The code and models are available at https://github.com/sdroh1027/DiffusionVID.","url_abs":"https://doi.org/10.1109/ACCESS.2023.3328341","url_pdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10299639","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diffusionvid-denoising-object-boxes-with","repo_url":"https://github.com/sdroh1027/DiffusionVID","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"video-object-detection","task_name":"Video Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-object-detection-on-imagenet-vid","task":"Video Object Detection","dataset":"ImageNet VID","model":"DiffusionVID (Swin-B)","rank_in_archive_order":2,"of":33,"metrics":{"MAP ":"92.5"},"uses_additional_data":false},{"leaderboard":"/sota/video-object-detection-on-imagenet-vid","task":"Video Object Detection","dataset":"ImageNet VID","model":"DiffusionVID (ResNet-101)","rank_in_archive_order":11,"of":33,"metrics":{"MAP ":"87.1"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}