{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mdcs-more-diverse-experts-with-consistency","title":"MDCS: More Diverse Experts with Consistency Self-distillation for Long-tailed Recognition","arxiv_id":"2308.09922","date":"2023-08-19","proceeding":"ICCV 2023 1","authors":["QiHao Zhao","Chen Jiang","Wei Hu","Fan Zhang","Jun Liu"],"abstract":"Recently, multi-expert methods have led to significant improvements in long-tail recognition (LTR). We summarize two aspects that need further enhancement to contribute to LTR boosting: (1) More diverse experts; (2) Lower model variance. However, the previous methods didn't handle them well. To this end, we propose More Diverse experts with Consistency Self-distillation (MDCS) to bridge the gap left by earlier methods. Our MDCS approach consists of two core components: Diversity Loss (DL) and Consistency Self-distillation (CS). In detail, DL promotes diversity among experts by controlling their focus on different categories. To reduce the model variance, we employ KL divergence to distill the richer knowledge of weakly augmented instances for the experts' self-distillation. In particular, we design Confident Instance Sampling (CIS) to select the correctly classified instances for CS to avoid biased/noisy knowledge. In the analysis and ablation study, we demonstrate that our method compared with previous work can effectively increase the diversity of experts, significantly reduce the variance of the model, and improve recognition accuracy. Moreover, the roles of our DL and CS are mutually reinforcing and coupled: the diversity of experts benefits from the CS, and the CS cannot achieve remarkable results without the DL. Experiments show our MDCS outperforms the state-of-the-art by 1% $\\sim$ 2% on five popular long-tailed benchmarks, including CIFAR10-LT, CIFAR100-LT, ImageNet-LT, Places-LT, and iNaturalist 2018. The code is available at https://github.com/fistyee/MDCS.","url_abs":"https://arxiv.org/abs/2308.09922v2","url_pdf":"https://arxiv.org/pdf/2308.09922v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mdcs-more-diverse-experts-with-consistency","repo_url":"https://github.com/fistyee/mdcs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"long-tail-learning","task_name":"Long-tail Learning"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/long-tail-learning-on-cifar-10-lt-r-50","task":"Long-tail Learning","dataset":"CIFAR-10-LT (ρ=50)","model":"MDCS","rank_in_archive_order":5,"of":8,"metrics":{"Error Rate":"11.7"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-cifar-100-lt-r-100","task":"Long-tail Learning","dataset":"CIFAR-100-LT (ρ=100)","model":"MDCS","rank_in_archive_order":10,"of":66,"metrics":{"Error Rate":"43.9"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-cifar-100-lt-r-50","task":"Long-tail Learning","dataset":"CIFAR-100-LT (ρ=50)","model":"MDCS","rank_in_archive_order":9,"of":25,"metrics":{"Error Rate":"39.9"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-imagenet-lt","task":"Long-tail Learning","dataset":"ImageNet-LT","model":"MDCS (ResNeXt-50)","rank_in_archive_order":12,"of":69,"metrics":{"Top-1 Accuracy":"61.8"},"uses_additional_data":false},{"leaderboard":"/sota/long-tail-learning-on-inaturalist-2018","task":"Long-tail Learning","dataset":"iNaturalist 2018","model":"MDCS(Resnet50)","rank_in_archive_order":11,"of":43,"metrics":{"Top-1 Accuracy":"75.6%"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.09922","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.09922"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fistyee/mdcs","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/fistyee/MDCS","reach":{"status":"ok"}}],"summary":{"ran":6,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"dc8b3c5ea5f84173","entry":"MDCSLoss","repo":"fistyee/MDCS","repo_kind":"official","path":"model/loss.py","file_url":"https://github.com/fistyee/MDCS/blob/HEAD/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc8b3c5ea5f84173"}},{"code_sha256_prefix":"1c1fdcd838303a0d","entry":"accuracy","repo":"fistyee/mdcs","repo_kind":"official","path":"model/metric.py","file_url":"https://github.com/fistyee/mdcs/blob/HEAD/model/metric.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c1fdcd838303a0d"}},{"code_sha256_prefix":"09b93518f9edc2dc","entry":"cat_mask","repo":"fistyee/mdcs","repo_kind":"official","path":"model/loss.py","file_url":"https://github.com/fistyee/mdcs/blob/HEAD/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09b93518f9edc2dc"}},{"code_sha256_prefix":"5f5ee5e8c168b306","entry":"dkd_loss","repo":"fistyee/mdcs","repo_kind":"official","path":"model/loss.py","file_url":"https://github.com/fistyee/mdcs/blob/HEAD/model/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f5ee5e8c168b306"}},{"code_sha256_prefix":"4dcf06ba28983266","entry":"focal_loss","repo":"fistyee/mdcs","repo_kind":"official","path":"model/loss.py","file_url":"https://github.com/fistyee/mdcs/blob/HEAD/model/loss.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4dcf06ba28983266"}},{"code_sha256_prefix":"23198d10b9f73030","entry":"learing_rate_scheduler","repo":"fistyee/mdcs","repo_kind":"official","path":"train_places.py","file_url":"https://github.com/fistyee/mdcs/blob/HEAD/train_places.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"23198d10b9f73030"}},{"code_sha256_prefix":"c642158604bebe8a","entry":"top_k_acc","repo":"fistyee/mdcs","repo_kind":"official","path":"model/metric.py","file_url":"https://github.com/fistyee/mdcs/blob/HEAD/model/metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c642158604bebe8a"}},{"code_sha256_prefix":"0933d643f92c12aa","entry":"init_weights","repo":"fistyee/mdcs","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/fistyee/mdcs/blob/HEAD/model/model.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0933d643f92c12aa"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}