{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/masked-frequency-modeling-for-self-supervised","title":"Masked Frequency Modeling for Self-Supervised Visual Pre-Training","arxiv_id":"2206.07706","date":"2022-06-15","proceeding":null,"authors":["Jiahao Xie","Wei Li","Xiaohang Zhan","Ziwei Liu","Yew Soon Ong","Chen Change Loy"],"abstract":"We present Masked Frequency Modeling (MFM), a unified frequency-domain-based approach for self-supervised pre-training of visual models. Instead of randomly inserting mask tokens to the input embeddings in the spatial domain, in this paper, we shift the perspective to the frequency domain. Specifically, MFM first masks out a portion of frequency components of the input image and then predicts the missing frequencies on the frequency spectrum. Our key insight is that predicting masked components in the frequency domain is more ideal to reveal underlying image patterns rather than predicting masked patches in the spatial domain, due to the heavy spatial redundancy. Our findings suggest that with the right configuration of mask-and-predict strategy, both the structural information within high-frequency components and the low-level statistics among low-frequency counterparts are useful in learning good representations. For the first time, MFM demonstrates that, for both ViT and CNN, a simple non-Siamese framework can learn meaningful representations even using none of the following: (i) extra data, (ii) extra model, (iii) mask token. Experimental results on image classification and semantic segmentation, as well as several robustness benchmarks show the competitive performance and advanced robustness of MFM compared with recent masked image modeling approaches. Furthermore, we also comprehensively investigate the effectiveness of classical image restoration tasks for representation learning from a unified frequency perspective and reveal their intriguing relations with our MFM approach.","url_abs":"https://arxiv.org/abs/2206.07706v2","url_pdf":"https://arxiv.org/pdf/2206.07706v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"masked-frequency-modeling-for-self-supervised","repo_url":"https://github.com/jiahao000/mfm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"masked-frequency-modeling-for-self-supervised","repo_url":"https://github.com/CoinCheung/MFM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"masked-frequency-modeling-for-self-supervised","repo_url":"https://github.com/weivision/correlational-image-modeling","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-restoration","task_name":"Image Restoration"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.07706","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.07706"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/weivision/correlational-image-modeling","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/CoinCheung/MFM","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiahao000/mfm","reach":null}],"summary":{"ran":2,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1},"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"366dccf153cf011c","entry":"FrequencyLoss","repo":"jiahao000/mfm","repo_kind":"official","path":"models/mfm.py","file_url":"https://github.com/jiahao000/mfm/blob/HEAD/models/mfm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"366dccf153cf011c"}},{"code_sha256_prefix":"5d032f2f9b215394","entry":"MaskedFFT","repo":"CoinCheung/MFM","repo_kind":"listed","path":"mfm/fft_masker.py","file_url":"https://github.com/CoinCheung/MFM/blob/HEAD/mfm/fft_masker.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d032f2f9b215394"}},{"code_sha256_prefix":"0fba080ff4842e62","entry":"get_args_parser","repo":"CoinCheung/MFM","repo_kind":"listed","path":"train_mfm.py","file_url":"https://github.com/CoinCheung/MFM/blob/HEAD/train_mfm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0fba080ff4842e62"}},{"code_sha256_prefix":"d9b018a98952dfd4","entry":"parse_batch","repo":"CoinCheung/MFM","repo_kind":"listed","path":"train_mfm.py","file_url":"https://github.com/CoinCheung/MFM/blob/HEAD/train_mfm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9b018a98952dfd4"}},{"code_sha256_prefix":"5d82922887d30a98","entry":"MFM","repo":"jiahao000/mfm","repo_kind":"official","path":"models/mfm.py","file_url":"https://github.com/jiahao000/mfm/blob/HEAD/models/mfm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"5d82922887d30a98"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}