Papers › Explanation-Aware Experience Replay in Rule-Dense Environments

Explanation-Aware Experience Replay in Rule-Dense Environments

29 Sep 2021arXiv:2109.14711archive 2025-07-28

Francesco Sovrano, Alex Raymond, Amanda Prorok

Human environments are often regulated by explicit and complex rulesets. Integrating Reinforcement Learning (RL) agents into such environments motivates the development of learning mechanisms that perform well in rule-dense and exception-ridden environments such as autonomous driving on regulated roads. In this paper, we propose a method for organising experience by means of partitioning the experience buffer into clusters labelled on a per-explanation basis. We present discrete and continuous navigation environments compatible with modular rulesets and 9 learning tasks. For environments with explainable rulesets, we convert rule-based explanations into case-based explanations by allocating state-transitions into clusters labelled with explanations. This allows us to sample experiences in a curricular and task-oriented manner, focusing on the rarity, importance, and meaning of events. We label this concept Explanation-Awareness (XA). We perform XA experience replay (XAER) with intra and inter-cluster prioritisation, and introduce XA-compatible versions of DQN, TD3, and SAC. Performance is consistently superior with XA versions of those algorithms, compared to traditional Prioritised Experience Replay baselines, indicating that explanation engineering can be used in lieu of reward engineering for environments with explainable features.

Paper PDF Code

Code

proroklab/xaer officialmentioned in papermentioned on GitHubtf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Autonomous Driving Reinforcement Learning (RL)

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

1x1 Convolution Adam Average Pooling Clipped Double Q-learning Convolution DQN Dense Connections Dilated Convolution Experience Replay Global Average Pooling Q-Learning ReLU SAC TD3 Target Policy Smoothing

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections