{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/catch-missing-details-image-reconstruction","title":"Catch Missing Details: Image Reconstruction with Frequency Augmented Variational Autoencoder","arxiv_id":"2305.02541","date":"2023-05-04","proceeding":"CVPR 2023 1","authors":["Xinmiao Lin","Yikang Li","Jenhao Hsiao","Chiuman Ho","Yu Kong"],"abstract":"The popular VQ-VAE models reconstruct images through learning a discrete codebook but suffer from a significant issue in the rapid quality degradation of image reconstruction as the compression rate rises. One major reason is that a higher compression rate induces more loss of visual signals on the higher frequency spectrum which reflect the details on pixel space. In this paper, a Frequency Complement Module (FCM) architecture is proposed to capture the missing frequency information for enhancing reconstruction quality. The FCM can be easily incorporated into the VQ-VAE structure, and we refer to the new model as Frequency Augmented VAE (FA-VAE). In addition, a Dynamic Spectrum Loss (DSL) is introduced to guide the FCMs to balance between various frequencies dynamically for optimal reconstruction. FA-VAE is further extended to the text-to-image synthesis task, and a Cross-attention Autoregressive Transformer (CAT) is proposed to obtain more precise semantic attributes in texts. Extensive reconstruction experiments with different compression rates are conducted on several benchmark datasets, and the results demonstrate that the proposed FA-VAE is able to restore more faithfully the details compared to SOTA methods. CAT also shows improved generation quality with better image-text semantic alignment.","url_abs":"https://arxiv.org/abs/2305.02541v2","url_pdf":"https://arxiv.org/pdf/2305.02541v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"catch-missing-details-image-reconstruction","repo_url":"https://github.com/oppo-us-research/FA-VAE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"vq-vae","method_name":"VQ-VAE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.02541","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.02541"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/oppo-us-research/FA-VAE","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":3,"ran_draft_wrong":2,"unverified":10},"by_repo_kind":{"official":{"samples":15,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"models/gpt_ca.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/models/gpt_ca.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"4cb732f513d69dfd","entry":"disabled_train","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"models/txt_cond_transformer.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/models/txt_cond_transformer.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb732f513d69dfd"}},{"code_sha256_prefix":"c16aee9490eb8729","entry":"eval_decorator","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"models/gpt_ca.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/models/gpt_ca.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c16aee9490eb8729"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"models/gpt_ca.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/models/gpt_ca.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"eca6cdf05972a95e","entry":"l2norm","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"models/l2_quantize.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/models/l2_quantize.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eca6cdf05972a95e"}},{"code_sha256_prefix":"a6e226dbe417263a","entry":"build_model","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"CLIP/clip/model.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/CLIP/clip/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6e226dbe417263a"}},{"code_sha256_prefix":"857a5f9aa43e2cf3","entry":"configure_optimizer","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"utils.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"857a5f9aa43e2cf3"}},{"code_sha256_prefix":"8e9b7f9a512aa375","entry":"hinge_d_loss","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"losses/hinge.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/losses/hinge.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e9b7f9a512aa375"}},{"code_sha256_prefix":"e3931d2003beff13","entry":"hinge_d_loss","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"models/discriminator.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/models/discriminator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e3931d2003beff13"}},{"code_sha256_prefix":"e7a8d15fe9e14a90","entry":"hinge_g_loss","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"losses/hinge.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/losses/hinge.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e7a8d15fe9e14a90"}},{"code_sha256_prefix":"50984532271d504f","entry":"least_square_d_loss","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"models/discriminator.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/models/discriminator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"50984532271d504f"}},{"code_sha256_prefix":"a369f81e1d3a417f","entry":"recon_ffl_features_loss","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"losses/vqgan_losses.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/losses/vqgan_losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a369f81e1d3a417f"}},{"code_sha256_prefix":"c4367203aea0f7c1","entry":"recon_ffl_loss","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"losses/vqgan_losses.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/losses/vqgan_losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c4367203aea0f7c1"}},{"code_sha256_prefix":"eeed9bf48de59e7b","entry":"recon_sl_gaussian_features_loss","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"losses/vqgan_losses.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/losses/vqgan_losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eeed9bf48de59e7b"}},{"code_sha256_prefix":"1c3cd7aa5e8128bd","entry":"vanilla_d_loss","repo":"oppo-us-research/FA-VAE","repo_kind":"official","path":"models/discriminator.py","file_url":"https://github.com/oppo-us-research/FA-VAE/blob/HEAD/models/discriminator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c3cd7aa5e8128bd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}