Papers › Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in...

Adaptive Action Duration with Contextual Bandits for Deep Reinforcement Learning in Dynamic Environments

17 Jun 2025arXiv:2507.00030archive 2025-07-28

Abhishek Verma, Nallarasan V, Balaraman Ravindran

Deep Reinforcement Learning (DRL) has achieved remarkable success in complex sequential decision-making tasks, such as playing Atari 2600 games and mastering board games. A critical yet underexplored aspect of DRL is the temporal scale of action execution. We propose a novel paradigm that integrates contextual bandits with DRL to adaptively select action durations, enhancing policy flexibility and computational efficiency. Our approach augments a Deep Q-Network (DQN) with a contextual bandit module that learns to choose optimal action repetition rates based on state contexts. Experiments on Atari 2600 games demonstrate significant performance improvements over static duration baselines, highlighting the efficacy of adaptive temporal abstractions in DRL. This paradigm offers a scalable solution for real-time applications like gaming and robotics, where dynamic action durations are critical.

Paper PDF Code

Code

abhi460729/Action-Duration-with-Contextual-Bandits-for-Deep-Reinforcement-Learning-in-Dynamic-Environments officialpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Atari Games Board Games Computational Efficiency Decision Making Deep Learning Deep Reinforcement Learning Multi-Armed Bandits Reinforcement Learning Reinforcement Learning (Atari Games)Sequential Decision Making

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Reinforcement Learning (Atari Games)	Seaquest - OpenAI Gym	BanditDQN	Average Return	87 ± 45	#1 of 1	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections