{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/collaborative-unsupervised-visual","title":"Collaborative Unsupervised Visual Representation Learning from Decentralized Data","arxiv_id":"2108.06492","date":"2021-08-14","proceeding":"ICCV 2021 10","authors":["Weiming Zhuang","Xin Gan","Yonggang Wen","Shuai Zhang","Shuai Yi"],"abstract":"Unsupervised representation learning has achieved outstanding performances using centralized data available on the Internet. However, the increasing awareness of privacy protection limits sharing of decentralized unlabeled image data that grows explosively in multiple parties (e.g., mobile phones and cameras). As such, a natural problem is how to leverage these data to learn visual representations for downstream tasks while preserving data privacy. To address this problem, we propose a novel federated unsupervised learning framework, FedU. In this framework, each party trains models from unlabeled data independently using contrastive learning with an online network and a target network. Then, a central server aggregates trained models and updates clients' models with the aggregated model. It preserves data privacy as each party only has access to its raw data. Decentralized data among multiple parties are normally non-independent and identically distributed (non-IID), leading to performance degradation. To tackle this challenge, we propose two simple but effective methods: 1) We design the communication protocol to upload only the encoders of online networks for server aggregation and update them with the aggregated encoder; 2) We introduce a new module to dynamically decide how to update predictors based on the divergence caused by non-IID. The predictor is the other component of the online network. Extensive experiments and ablations demonstrate the effectiveness and significance of FedU. It outperforms training with only one party by over 5% and other methods by over 14% in linear and semi-supervised evaluation on non-IID data.","url_abs":"https://arxiv.org/abs/2108.06492v1","url_pdf":"https://arxiv.org/pdf/2108.06492v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"collaborative-unsupervised-visual","repo_url":"https://github.com/EasyFL-AI/EasyFL/tree/master/applications/fedssl","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"federated-learning","task_name":"Federated Learning"},{"task_slug":"federated-unsupervised-learning","task_name":"Federated Unsupervised Learning"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2108.06492","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.06492"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/EasyFL-AI/EasyFL/tree/master/applications/fedssl","reach":null}],"summary":{"ran":2,"unverified":3},"by_repo_kind":{"official":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"43b65445bde2bf33","entry":"MLP","repo":"EasyFL-AI/EasyFL","repo_kind":"official","path":"applications/fedssl/model.py","file_url":"https://github.com/EasyFL-AI/EasyFL/blob/HEAD/applications/fedssl/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"43b65445bde2bf33"}},{"code_sha256_prefix":"75992b6b253f9430","entry":"ResNet18","repo":"EasyFL-AI/EasyFL","repo_kind":"official","path":"applications/fedssl/model.py","file_url":"https://github.com/EasyFL-AI/EasyFL/blob/HEAD/applications/fedssl/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"75992b6b253f9430"}},{"code_sha256_prefix":"4a08683ecb8da2f1","entry":"BaseModel","repo":"EasyFL-AI/EasyFL","repo_kind":"official","path":"applications/fedssl/model.py","file_url":"https://github.com/EasyFL-AI/EasyFL/blob/HEAD/applications/fedssl/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4a08683ecb8da2f1"}},{"code_sha256_prefix":"279410413f7add8f","entry":"MoCoModel","repo":"EasyFL-AI/EasyFL","repo_kind":"official","path":"applications/fedssl/model.py","file_url":"https://github.com/EasyFL-AI/EasyFL/blob/HEAD/applications/fedssl/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"279410413f7add8f"}},{"code_sha256_prefix":"73958f1ff0c64689","entry":"ResNet","repo":"EasyFL-AI/EasyFL","repo_kind":"official","path":"applications/fedssl/model.py","file_url":"https://github.com/EasyFL-AI/EasyFL/blob/HEAD/applications/fedssl/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"73958f1ff0c64689"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}