{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/calibration-matters-tackling-maximization","title":"Calibration Matters: Tackling Maximization Bias in Large-scale Advertising Recommendation Systems","arxiv_id":"2205.09809","date":"2022-05-19","proceeding":null,"authors":["Yewen Fan","Nian Si","Kun Zhang"],"abstract":"Calibration is defined as the ratio of the average predicted click rate to the true click rate. The optimization of calibration is essential to many online advertising recommendation systems because it directly affects the downstream bids in ads auctions and the amount of money charged to advertisers. Despite its importance, calibration optimization often suffers from a problem called \"maximization bias\". Maximization bias refers to the phenomenon that the maximum of predicted values overestimates the true maximum. The problem is introduced because the calibration is computed on the set selected by the prediction model itself. It persists even if unbiased predictions can be achieved on every datapoint and worsens when covariate shifts exist between the training and test sets. To mitigate this problem, we theorize the quantification of maximization bias and propose a variance-adjusting debiasing (VAD) meta-algorithm in this paper. The algorithm is efficient, robust, and practical as it is able to mitigate maximization bias problems under covariate shifts, neither incurring additional online serving costs nor compromising the ranking performance. We demonstrate the effectiveness of the proposed algorithm using a state-of-the-art recommendation neural network model on a large-scale real-world dataset.","url_abs":"https://arxiv.org/abs/2205.09809v5","url_pdf":"https://arxiv.org/pdf/2205.09809v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"calibration-matters-tackling-maximization","repo_url":"https://github.com/tofuwen/vad","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2205.09809","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.09809"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tofuwen/vad","reach":null}],"summary":{"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":4,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"72e28f0796a996d5","entry":"calculate_lambda","repo":"tofuwen/vad","repo_kind":"official","path":"VAD_util.py","file_url":"https://github.com/tofuwen/vad/blob/HEAD/VAD_util.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72e28f0796a996d5"}},{"code_sha256_prefix":"d8506a2054d2633e","entry":"compute_calibration_error","repo":"tofuwen/vad","repo_kind":"official","path":"calibration_utils.py","file_url":"https://github.com/tofuwen/vad/blob/HEAD/calibration_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d8506a2054d2633e"}},{"code_sha256_prefix":"07a6304870ccbb63","entry":"create_binned_data","repo":"tofuwen/vad","repo_kind":"official","path":"calibration_utils.py","file_url":"https://github.com/tofuwen/vad/blob/HEAD/calibration_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"07a6304870ccbb63"}},{"code_sha256_prefix":"68231a514b8cc5f7","entry":"get_bin_boundaries","repo":"tofuwen/vad","repo_kind":"official","path":"calibration_utils.py","file_url":"https://github.com/tofuwen/vad/blob/HEAD/calibration_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"68231a514b8cc5f7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}