{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/policy-optimization-with-advantage","title":"Policy Optimization with Advantage Regularization for Long-Term Fairness in Decision Systems","arxiv_id":"2210.12546","date":"2022-10-22","proceeding":null,"authors":["Eric Yang Yu","Zhizhen Qin","Min Kyung Lee","Sicun Gao"],"abstract":"Long-term fairness is an important factor of consideration in designing and deploying learning-based decision systems in high-stake decision-making contexts. Recent work has proposed the use of Markov Decision Processes (MDPs) to formulate decision-making with long-term fairness requirements in dynamically changing environments, and demonstrated major challenges in directly deploying heuristic and rule-based policies that worked well in static environments. We show that policy optimization methods from deep reinforcement learning can be used to find strictly better decision policies that can often achieve both higher overall utility and less violation of the fairness requirements, compared to previously-known strategies. In particular, we propose new methods for imposing fairness requirements in policy optimization by regularizing the advantage evaluation of different actions. Our proposed methods make it easy to impose fairness constraints without reward engineering or sacrificing training efficiency. We perform detailed analyses in three established case studies, including attention allocation in incident monitoring, bank loan approval, and vaccine distribution in population networks.","url_abs":"https://arxiv.org/abs/2210.12546v1","url_pdf":"https://arxiv.org/pdf/2210.12546v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"policy-optimization-with-advantage","repo_url":"https://github.com/ericyangyu/pocar","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.12546","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.12546"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ericyangyu/pocar","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"11dd4986e15b6ffb","entry":"build_diag_gauss_policy","repo":"ericyangyu/pocar","repo_kind":"official","path":"attention_allocation_experiment/agents/cpo/models.py","file_url":"https://github.com/ericyangyu/pocar/blob/HEAD/attention_allocation_experiment/agents/cpo/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11dd4986e15b6ffb"}},{"code_sha256_prefix":"c48048f866590296","entry":"build_layers","repo":"ericyangyu/pocar","repo_kind":"official","path":"attention_allocation_experiment/agents/cpo/models.py","file_url":"https://github.com/ericyangyu/pocar/blob/HEAD/attention_allocation_experiment/agents/cpo/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c48048f866590296"}},{"code_sha256_prefix":"aa00123df64953ed","entry":"build_mlp","repo":"ericyangyu/pocar","repo_kind":"official","path":"attention_allocation_experiment/agents/cpo/models.py","file_url":"https://github.com/ericyangyu/pocar/blob/HEAD/attention_allocation_experiment/agents/cpo/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa00123df64953ed"}},{"code_sha256_prefix":"bcf2a2e85422f10d","entry":"validate_reward","repo":"ericyangyu/pocar","repo_kind":"official","path":"attention_allocation_experiment/environments/core.py","file_url":"https://github.com/ericyangyu/pocar/blob/HEAD/attention_allocation_experiment/environments/core.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bcf2a2e85422f10d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}