{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/slot-order-matters-for-compositional-scene","title":"Towards Improving the Generation Quality of Autoregressive Slot VAEs","arxiv_id":"2206.01370","date":"2022-06-03","proceeding":null,"authors":["Patrick Emami","Pan He","Sanjay Ranka","Anand Rangarajan"],"abstract":"Unconditional scene inference and generation are challenging to learn jointly with a single compositional model. Despite encouraging progress on models that extract object-centric representations (''slots'') from images, unconditional generation of scenes from slots has received less attention. This is primarily because learning the multi-object relations necessary to imagine coherent scenes is difficult. We hypothesize that most existing slot-based models have a limited ability to learn object correlations. We propose two improvements that strengthen object correlation learning. The first is to condition the slots on a global, scene-level variable that captures higher-order correlations between slots. Second, we address the fundamental lack of a canonical order for objects in images by proposing to learn a consistent order to use for the autoregressive generation of scene objects. Specifically, we train an autoregressive slot prior to sequentially generate scene objects following a learned order. Ordered slot inference entails first estimating a randomly ordered set of slots using existing approaches for extracting slots from images, then aligning those slots to ordered slots generated autoregressively with the slot prior. Our experiments across three multi-object environments demonstrate clear gains in unconditional scene generation quality. Detailed ablation studies are also provided that validate the two proposed improvements.","url_abs":"https://arxiv.org/abs/2206.01370v3","url_pdf":"https://arxiv.org/pdf/2206.01370v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"slot-order-matters-for-compositional-scene","repo_url":"https://github.com/pemami4911/segregate-relate-imagine","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"object","task_name":"Object"},{"task_slug":"scene-generation","task_name":"Scene Generation"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"variational-inference","task_name":"Variational Inference"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"variational-inference","method_name":"Variational Inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.01370","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.01370"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pemami4911/segregate-relate-imagine","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bb452191fb5c6d51","entry":"adjusted_rand_index","repo":"pemami4911/segregate-relate-imagine","repo_kind":"official","path":"sri/metrics.py","file_url":"https://github.com/pemami4911/segregate-relate-imagine/blob/HEAD/sri/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bb452191fb5c6d51"}},{"code_sha256_prefix":"e89ddfc78bb6017c","entry":"colour_seg_masks","repo":"pemami4911/segregate-relate-imagine","repo_kind":"official","path":"sri/visualization.py","file_url":"https://github.com/pemami4911/segregate-relate-imagine/blob/HEAD/sri/visualization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e89ddfc78bb6017c"}},{"code_sha256_prefix":"26004642f3667955","entry":"create_mask_image","repo":"pemami4911/segregate-relate-imagine","repo_kind":"official","path":"sri/visualization.py","file_url":"https://github.com/pemami4911/segregate-relate-imagine/blob/HEAD/sri/visualization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26004642f3667955"}},{"code_sha256_prefix":"607c63ed17c02704","entry":"get_mask_plot_colors","repo":"pemami4911/segregate-relate-imagine","repo_kind":"official","path":"sri/visualization.py","file_url":"https://github.com/pemami4911/segregate-relate-imagine/blob/HEAD/sri/visualization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"607c63ed17c02704"}},{"code_sha256_prefix":"8439bd65f5ac1c20","entry":"mvn","repo":"pemami4911/segregate-relate-imagine","repo_kind":"official","path":"sri/utils.py","file_url":"https://github.com/pemami4911/segregate-relate-imagine/blob/HEAD/sri/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8439bd65f5ac1c20"}},{"code_sha256_prefix":"1e58c4d04b02b286","entry":"pixel_mse","repo":"pemami4911/segregate-relate-imagine","repo_kind":"official","path":"sri/metrics.py","file_url":"https://github.com/pemami4911/segregate-relate-imagine/blob/HEAD/sri/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1e58c4d04b02b286"}},{"code_sha256_prefix":"15315189e1f1c9db","entry":"std_mvn","repo":"pemami4911/segregate-relate-imagine","repo_kind":"official","path":"sri/utils.py","file_url":"https://github.com/pemami4911/segregate-relate-imagine/blob/HEAD/sri/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"15315189e1f1c9db"}},{"code_sha256_prefix":"aacd6f716b035290","entry":"tensor_to_png","repo":"pemami4911/segregate-relate-imagine","repo_kind":"official","path":"compute_fid.py","file_url":"https://github.com/pemami4911/segregate-relate-imagine/blob/HEAD/compute_fid.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aacd6f716b035290"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}