{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/training-effective-ensemble-on-imbalanced","title":"Self-paced Ensemble for Highly Imbalanced Massive Data Classification","arxiv_id":"1909.03500","date":"2019-09-08","proceeding":null,"authors":["Zhining Liu","Wei Cao","Zhifeng Gao","Jiang Bian","Hechang Chen","Yi Chang","Tie-Yan Liu"],"abstract":"Many real-world applications reveal difficulties in learning classifiers from imbalanced data. The rising big data era has been witnessing more classification tasks with large-scale but extremely imbalance and low-quality datasets. Most of existing learning methods suffer from poor performance or low computation efficiency under such a scenario. To tackle this problem, we conduct deep investigations into the nature of class imbalance, which reveals that not only the disproportion between classes, but also other difficulties embedded in the nature of data, especially, noises and class overlapping, prevent us from learning effective classifiers. Taking those factors into consideration, we propose a novel framework for imbalance classification that aims to generate a strong ensemble by self-paced harmonizing data hardness via under-sampling. Extensive experiments have shown that this new framework, while being very computationally efficient, can lead to robust performance even under highly overlapping classes and extremely skewed distribution. Note that, our methods can be easily adapted to most of existing learning methods (e.g., C4.5, SVM, GBDT and Neural Network) to boost their performance on imbalanced data.","url_abs":"https://arxiv.org/abs/1909.03500v3","url_pdf":"https://arxiv.org/pdf/1909.03500v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"training-effective-ensemble-on-imbalanced","repo_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"imbalanced-classification","task_name":"imbalanced classification"}],"methods":[{"method_slug":"svm","method_name":"SVM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1909.03500","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1909.03500"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ZhiningLiu1998/self-paced-ensemble","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"unverified":12},"by_repo_kind":{"official":{"samples":13,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8dc1dbc6e2b2e261","entry":"check_target_type","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_validation.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_validation.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8dc1dbc6e2b2e261"}},{"code_sha256_prefix":"fe7f7070ccd65a42","entry":"check_eval_datasets","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_validation_data.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_validation_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe7f7070ccd65a42"}},{"code_sha256_prefix":"191e69f2dc4a7ca7","entry":"check_n_target_samples","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_validation_param.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_validation_param.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"191e69f2dc4a7ca7"}},{"code_sha256_prefix":"a6714376095e7577","entry":"check_sampling_strategy","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_validation.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_validation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6714376095e7577"}},{"code_sha256_prefix":"0d7d1a79c88c9f3d","entry":"check_target_label","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_validation_param.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_validation_param.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0d7d1a79c88c9f3d"}},{"code_sha256_prefix":"b174867866f9c61e","entry":"check_target_label_and_n_target_samples","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_validation_param.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_validation_param.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b174867866f9c61e"}},{"code_sha256_prefix":"73c1261f7af67bdb","entry":"imbalance_train_test_split","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_utils.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"73c1261f7af67bdb"}},{"code_sha256_prefix":"54be56bd49600894","entry":"load_covtype_dataset","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_utils.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54be56bd49600894"}},{"code_sha256_prefix":"7128f053169a319a","entry":"make_binary_classification_target","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_utils.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7128f053169a319a"}},{"code_sha256_prefix":"1c0a7146d1bf4c80","entry":"plot_class_distribution","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_plot.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c0a7146d1bf4c80"}},{"code_sha256_prefix":"8684fa95672d989e","entry":"plot_scatter","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_plot.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8684fa95672d989e"}},{"code_sha256_prefix":"5feff5207339c493","entry":"set_ax_border","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/utils/_plot.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/utils/_plot.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5feff5207339c493"}},{"code_sha256_prefix":"725d4912711bd963","entry":"sort_dict_by_key","repo":"ZhiningLiu1998/self-paced-ensemble","repo_kind":"official","path":"self_paced_ensemble/self_paced_ensemble/base.py","file_url":"https://github.com/ZhiningLiu1998/self-paced-ensemble/blob/HEAD/self_paced_ensemble/self_paced_ensemble/base.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"725d4912711bd963"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}