{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-complement-and-to-defer-to","title":"Learning to Complement and to Defer to Multiple Users","arxiv_id":"2407.07003","date":"2024-07-09","proceeding":null,"authors":["Zheng Zhang","Wenjie Ai","Kevin Wells","David Rosewarne","Thanh-Toan Do","Gustavo Carneiro"],"abstract":"With the development of Human-AI Collaboration in Classification (HAI-CC), integrating users and AI predictions becomes challenging due to the complex decision-making process. This process has three options: 1) AI autonomously classifies, 2) learning to complement, where AI collaborates with users, and 3) learning to defer, where AI defers to users. Despite their interconnected nature, these options have been studied in isolation rather than as components of a unified system. In this paper, we address this weakness with the novel HAI-CC methodology, called Learning to Complement and to Defer to Multiple Users (LECODU). LECODU not only combines learning to complement and learning to defer strategies, but it also incorporates an estimation of the optimal number of users to engage in the decision process. The training of LECODU maximises classification accuracy and minimises collaboration costs associated with user involvement. Comprehensive evaluations across real-world and synthesized datasets demonstrate LECODU's superior performance compared to state-of-the-art HAI-CC methods. Remarkably, even when relying on unreliable users with high rates of label noise, LECODU exhibits significant improvement over both human decision-makers alone and AI alone.","url_abs":"https://arxiv.org/abs/2407.07003v1","url_pdf":"https://arxiv.org/pdf/2407.07003v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-complement-and-to-defer-to","repo_url":"https://github.com/zhengzhang37/lecodu","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.07003","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.07003"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhengzhang37/lecodu","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":5,"ran":14,"unverified":2},"by_repo_kind":{"official":{"samples":21,"ran":19,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":21,"samples":[{"code_sha256_prefix":"4a0ac9d611e018bc","entry":"AutoContrast","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/randaug.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/randaug.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4a0ac9d611e018bc"}},{"code_sha256_prefix":"61ed09bbdb8adb72","entry":"Brightness","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/randaug.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/randaug.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61ed09bbdb8adb72"}},{"code_sha256_prefix":"5e4ee5c45933ae09","entry":"Color","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/randaug.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/randaug.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e4ee5c45933ae09"}},{"code_sha256_prefix":"7b20f1288903c47c","entry":"ResNet18","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"PreResNet.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/PreResNet.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b20f1288903c47c"}},{"code_sha256_prefix":"eee26fa0aef03470","entry":"ResNet34","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"PreResNet.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/PreResNet.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eee26fa0aef03470"}},{"code_sha256_prefix":"dc322bdafd96ffba","entry":"bias_initial","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc322bdafd96ffba"}},{"code_sha256_prefix":"583f9780bdd00a45","entry":"conv3x3","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"PreResNet.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/PreResNet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"583f9780bdd00a45"}},{"code_sha256_prefix":"0f217ad023a21a61","entry":"debias_output","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f217ad023a21a61"}},{"code_sha256_prefix":"59ea92571d058dba","entry":"debias_pl","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/utils.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"59ea92571d058dba"}},{"code_sha256_prefix":"79f12de984fe5e1c","entry":"fftfreqnd","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/fmix.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/fmix.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"79f12de984fe5e1c"}},{"code_sha256_prefix":"cf3259099e459c51","entry":"fit","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/tools.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cf3259099e459c51"}},{"code_sha256_prefix":"122df027ef4a7990","entry":"fmix_loss","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/fmix.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/fmix.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"122df027ef4a7990"}},{"code_sha256_prefix":"8f421a9c1829d948","entry":"get_instance_noisy_label","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/tools.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8f421a9c1829d948"}},{"code_sha256_prefix":"eb5c9156d8d5d9c4","entry":"get_spectrum","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/fmix.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/fmix.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb5c9156d8d5d9c4"}},{"code_sha256_prefix":"536c9719cc690836","entry":"norm","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"utils/tools.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/utils/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"536c9719cc690836"}},{"code_sha256_prefix":"7c913c414af7196f","entry":"sample_gumbel","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"lecodu_chaoyang.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/lecodu_chaoyang.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7c913c414af7196f"}},{"code_sha256_prefix":"ef49e82a7403eee1","entry":"unpickle","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"dataloader_cifar.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/dataloader_cifar.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ef49e82a7403eee1"}},{"code_sha256_prefix":"b66ffd33f592d1a5","entry":"value_to_one_hot","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"lecodu_chaoyang.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/lecodu_chaoyang.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b66ffd33f592d1a5"}},{"code_sha256_prefix":"deedc113c7bf508c","entry":"value_to_one_hot","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"train_cifar_multiusers.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/train_cifar_multiusers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"deedc113c7bf508c"}},{"code_sha256_prefix":"50ab7cb81d56353e","entry":"create_model","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"lecodu_chaoyang.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/lecodu_chaoyang.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"50ab7cb81d56353e"}},{"code_sha256_prefix":"50915e2234edf43b","entry":"create_model","repo":"zhengzhang37/lecodu","repo_kind":"official","path":"train_cifar_multiusers.py","file_url":"https://github.com/zhengzhang37/lecodu/blob/HEAD/train_cifar_multiusers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"50915e2234edf43b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}