{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/outpainting-by-queries","title":"Outpainting by Queries","arxiv_id":"2207.05312","date":"2022-07-12","proceeding":null,"authors":["Kai Yao","Penglei Gao","Xi Yang","Kaizhu Huang","Jie Sun","Rui Zhang"],"abstract":"Image outpainting, which is well studied with Convolution Neural Network (CNN) based framework, has recently drawn more attention in computer vision. However, CNNs rely on inherent inductive biases to achieve effective sample learning, which may degrade the performance ceiling. In this paper, motivated by the flexible self-attention mechanism with minimal inductive biases in transformer architecture, we reframe the generalised image outpainting problem as a patch-wise sequence-to-sequence autoregression problem, enabling query-based image outpainting. Specifically, we propose a novel hybrid vision-transformer-based encoder-decoder framework, named \\textbf{Query} \\textbf{O}utpainting \\textbf{TR}ansformer (\\textbf{QueryOTR}), for extrapolating visual context all-side around a given image. Patch-wise mode's global modeling capacity allows us to extrapolate images from the attention mechanism's query standpoint. A novel Query Expansion Module (QEM) is designed to integrate information from the predicted queries based on the encoder's output, hence accelerating the convergence of the pure transformer even with a relatively small dataset. To further enhance connectivity between each patch, the proposed Patch Smoothing Module (PSM) re-allocates and averages the overlapped regions, thus providing seamless predicted images. We experimentally show that QueryOTR could generate visually appealing results smoothly and realistically against the state-of-the-art image outpainting approaches.","url_abs":"https://arxiv.org/abs/2207.05312v1","url_pdf":"https://arxiv.org/pdf/2207.05312v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"outpainting-by-queries","repo_url":"https://github.com/kaiseem/queryotr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"image-outpainting","task_name":"Image Outpainting"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.05312","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.05312"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kaiseem/queryotr","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/Kaiseem/QueryOTR","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":2,"unverified":5},"by_repo_kind":{"official":{"samples":8,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b0bbabd6de30f612","entry":"DiffAugment","repo":"Kaiseem/QueryOTR","repo_kind":"official","path":"models/DiffAug.py","file_url":"https://github.com/Kaiseem/QueryOTR/blob/HEAD/models/DiffAug.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0bbabd6de30f612"}},{"code_sha256_prefix":"011230b2b9b8fb6f","entry":"rand_brightness","repo":"Kaiseem/QueryOTR","repo_kind":"official","path":"models/DiffAug.py","file_url":"https://github.com/Kaiseem/QueryOTR/blob/HEAD/models/DiffAug.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"011230b2b9b8fb6f"}},{"code_sha256_prefix":"5b0d8787e670fc63","entry":"rand_saturation","repo":"Kaiseem/QueryOTR","repo_kind":"official","path":"models/DiffAug.py","file_url":"https://github.com/Kaiseem/QueryOTR/blob/HEAD/models/DiffAug.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5b0d8787e670fc63"}},{"code_sha256_prefix":"cfd2694ff8eac454","entry":"denorm_img","repo":"Kaiseem/QueryOTR","repo_kind":"official","path":"engine.py","file_url":"https://github.com/Kaiseem/QueryOTR/blob/HEAD/engine.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cfd2694ff8eac454"}},{"code_sha256_prefix":"e74fb8b7e38e352c","entry":"denorm_img","repo":"Kaiseem/QueryOTR","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/Kaiseem/QueryOTR/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e74fb8b7e38e352c"}},{"code_sha256_prefix":"719cc078ef36535e","entry":"is_image_file","repo":"Kaiseem/QueryOTR","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/Kaiseem/QueryOTR/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"719cc078ef36535e"}},{"code_sha256_prefix":"cf5d1aa663fdbae3","entry":"make_dataset","repo":"Kaiseem/QueryOTR","repo_kind":"official","path":"datasets.py","file_url":"https://github.com/Kaiseem/QueryOTR/blob/HEAD/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cf5d1aa663fdbae3"}},{"code_sha256_prefix":"56f33697a551d9b8","entry":"vit_base_patch16","repo":"Kaiseem/QueryOTR","repo_kind":"official","path":"models/VIT_old.py","file_url":"https://github.com/Kaiseem/QueryOTR/blob/HEAD/models/VIT_old.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"56f33697a551d9b8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}