{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/autoregressive-video-generation-without","title":"Autoregressive Video Generation without Vector Quantization","arxiv_id":"2412.14169","date":"2024-12-18","proceeding":null,"authors":["Haoge Deng","Ting Pan","Haiwen Diao","Zhengxiong Luo","Yufeng Cui","Huchuan Lu","Shiguang Shan","Yonggang Qi","Xinlong Wang"],"abstract":"This paper presents a novel approach that enables autoregressive video generation with high efficiency. We propose to reformulate the video generation problem as a non-quantized autoregressive modeling of temporal frame-by-frame prediction and spatial set-by-set prediction. Unlike raster-scan prediction in prior autoregressive models or joint distribution modeling of fixed-length tokens in diffusion models, our approach maintains the causal property of GPT-style models for flexible in-context capabilities, while leveraging bidirectional modeling within individual frames for efficiency. With the proposed approach, we train a novel video autoregressive model without vector quantization, termed NOVA. Our results demonstrate that NOVA surpasses prior autoregressive video models in data efficiency, inference speed, visual fidelity, and video fluency, even with a much smaller model capacity, i.e., 0.6B parameters. NOVA also outperforms state-of-the-art image diffusion models in text-to-image generation tasks, with a significantly lower training cost. Additionally, NOVA generalizes well across extended video durations and enables diverse zero-shot applications in one unified model. Code and models are publicly available at https://github.com/baaivision/NOVA.","url_abs":"https://arxiv.org/abs/2412.14169v1","url_pdf":"https://arxiv.org/pdf/2412.14169v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"autoregressive-video-generation-without","repo_url":"https://github.com/baaivision/nova","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.14169","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.14169"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/baaivision/NOVA","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/baaivision/nova","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":19,"unverified":7},"by_repo_kind":{"official":{"samples":26,"ran":19,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"60c4cd0d5289ebda","entry":"AdaLayerNorm","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"60c4cd0d5289ebda"}},{"code_sha256_prefix":"85377e46ac705725","entry":"AdaLayerNormZero","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85377e46ac705725"}},{"code_sha256_prefix":"d627a66e082b5f70","entry":"Attention","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d627a66e082b5f70"}},{"code_sha256_prefix":"a6b18aefa9c30394","entry":"Block","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a6b18aefa9c30394"}},{"code_sha256_prefix":"9252010cba1de0c1","entry":"DiffusionBlock","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9252010cba1de0c1"}},{"code_sha256_prefix":"c06a16a8febb2446","entry":"DiffusionMLP","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c06a16a8febb2446"}},{"code_sha256_prefix":"028ca8089a24ed40","entry":"GuidanceScaler","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"028ca8089a24ed40"}},{"code_sha256_prefix":"876ccd77339d9625","entry":"LabelEmbed","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"876ccd77339d9625"}},{"code_sha256_prefix":"8785a25e456048ac","entry":"MLP","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8785a25e456048ac"}},{"code_sha256_prefix":"7a41a8cdb50f38e6","entry":"MaskEmbed","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7a41a8cdb50f38e6"}},{"code_sha256_prefix":"d1412392f9460cc0","entry":"MotionEmbed","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d1412392f9460cc0"}},{"code_sha256_prefix":"803f92207c86ec75","entry":"PatchEmbed","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"803f92207c86ec75"}},{"code_sha256_prefix":"14e66b780fcd44af","entry":"PosEmbed","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14e66b780fcd44af"}},{"code_sha256_prefix":"76ede3fc8a37f868","entry":"Projector","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"76ede3fc8a37f868"}},{"code_sha256_prefix":"1f02d40b2da1f5fe","entry":"Registry","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1f02d40b2da1f5fe"}},{"code_sha256_prefix":"5ec9478ced359867","entry":"RotaryEmbed3D","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5ec9478ced359867"}},{"code_sha256_prefix":"aa91355939326691","entry":"TextEmbed","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa91355939326691"}},{"code_sha256_prefix":"8103232d2040cad8","entry":"TimeCondEmbed","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8103232d2040cad8"}},{"code_sha256_prefix":"f36aa3555265a176","entry":"VideoPosEmbed","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f36aa3555265a176"}},{"code_sha256_prefix":"ade0129d1a45e0b9","entry":"FlexAttentionCausal2D","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ade0129d1a45e0b9"}},{"code_sha256_prefix":"97f6bd68a990166d","entry":"NOVATransformer3DModel","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"97f6bd68a990166d"}},{"code_sha256_prefix":"634c92dbdd87cfe9","entry":"Transformer3DModel","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"634c92dbdd87cfe9"}},{"code_sha256_prefix":"3a44328c8bf72fe3","entry":"VisionTransformer","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3a44328c8bf72fe3"}},{"code_sha256_prefix":"7d03ff4b172aacee","entry":"image_decoder","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7d03ff4b172aacee"}},{"code_sha256_prefix":"9dc11e1a48d18afe","entry":"image_encoder","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9dc11e1a48d18afe"}},{"code_sha256_prefix":"bb1ef5506cb864f5","entry":"video_encoder","repo":"baaivision/nova","repo_kind":"official","path":"diffnext/models/transformers/transformer_nova.py","file_url":"https://github.com/baaivision/nova/blob/HEAD/diffnext/models/transformers/transformer_nova.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bb1ef5506cb864f5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}