{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/addressing-the-long-term-impact-of-ml","title":"Addressing the Long-term Impact of ML Decisions via Policy Regret","arxiv_id":"2106.01325","date":"2021-06-02","proceeding":null,"authors":["David Lindner","Hoda Heidari","Andreas Krause"],"abstract":"Machine Learning (ML) increasingly informs the allocation of opportunities to individuals and communities in areas such as lending, education, employment, and beyond. Such decisions often impact their subjects' future characteristics and capabilities in an a priori unknown fashion. The decision-maker, therefore, faces exploration-exploitation dilemmas akin to those in multi-armed bandits. Following prior work, we model communities as arms. To capture the long-term effects of ML-based allocation decisions, we study a setting in which the reward from each arm evolves every time the decision-maker pulls that arm. We focus on reward functions that are initially increasing in the number of pulls but may become (and remain) decreasing after a certain point. We argue that an acceptable sequential allocation of opportunities must take an arm's potential for growth into account. We capture these considerations through the notion of policy regret, a much stronger notion than the often-studied external regret, and present an algorithm with provably sub-linear policy regret for sufficiently long time horizons. We empirically compare our algorithm with several baselines and find that it consistently outperforms them, in particular for long time horizons.","url_abs":"https://arxiv.org/abs/2106.01325v1","url_pdf":"https://arxiv.org/pdf/2106.01325v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"addressing-the-long-term-impact-of-ml","repo_url":"https://github.com/david-lindner/single-peaked-bandits","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"multi-armed-bandits","task_name":"Multi-Armed Bandits"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2106.01325","atlas_url":"https://app.syntology.ai/?focus=2106.01325","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.01325"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/david-lindner/single-peaked-bandits","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":12},"by_repo_kind":{"official":{"samples":12,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2ab227cb843ea874","entry":"cleanup_frame","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/fico_experiment/fico.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/fico_experiment/fico.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2ab227cb843ea874"}},{"code_sha256_prefix":"c2824956a3f0b6ed","entry":"convert_percentiles","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/fico_experiment/fico.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/fico_experiment/fico.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c2824956a3f0b6ed"}},{"code_sha256_prefix":"818babbd0e1ece49","entry":"cumulative_reward","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/helpers.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"818babbd0e1ece49"}},{"code_sha256_prefix":"33dbbd124c7d9cb2","entry":"exp1_f1","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/experiments.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"33dbbd124c7d9cb2"}},{"code_sha256_prefix":"f3806811d7e241bb","entry":"exp1_f2","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/experiments.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f3806811d7e241bb"}},{"code_sha256_prefix":"39742acae3c2c1bb","entry":"exp2_f1","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/experiments.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"39742acae3c2c1bb"}},{"code_sha256_prefix":"3c853c8d20825854","entry":"get_optimal_arm_constant_reward_bandit","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/solvers/optimal.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/solvers/optimal.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3c853c8d20825854"}},{"code_sha256_prefix":"aac98b644db36e6f","entry":"get_reward","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/fico_experiment/preprocessing.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/fico_experiment/preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aac98b644db36e6f"}},{"code_sha256_prefix":"2fe9714ffac9aaee","entry":"get_reward_for_policy","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/helpers.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2fe9714ffac9aaee"}},{"code_sha256_prefix":"cc9b948c14a71d4c","entry":"optimal_policy_constant_reward_bandit","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/solvers/optimal.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/solvers/optimal.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cc9b948c14a71d4c"}},{"code_sha256_prefix":"6b02eb4f5ff352d7","entry":"read_totals","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/fico_experiment/fico.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/fico_experiment/fico.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6b02eb4f5ff352d7"}},{"code_sha256_prefix":"c77500dfa8e1e8b8","entry":"sample_group_population","repo":"david-lindner/single-peaked-bandits","repo_kind":"official","path":"src/single_peaked_bandits/fico_experiment/preprocessing.py","file_url":"https://github.com/david-lindner/single-peaked-bandits/blob/HEAD/src/single_peaked_bandits/fico_experiment/preprocessing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c77500dfa8e1e8b8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}