Explainable Reinforcement Learning via Reward Decomposition

Paper: Explainable Reinforcement Learning via Reward Decomposition

This paper presents a way of enabling Reinforcement Learning agents to explain thier decisions. They assume a decomposed reward funciton, into user interpretable reward-types and use these components / types to generate explanation. Their first contribution is the drQ algorithm that they prove to converge to the optimal Q* as well optimal compoenent Q_c* (something that prior works on decomposed reward functions for Reinforcement Learning lacked). Further they introduce concecpts like Reward Difference Explanation - a set/ vector that shows which reward component supports which action and further reduce this through Minimal Sufficient Explanations to provide concise and to-the-point explanations.

Keywords :
Ex-RL, Explainable RL, Artificial Intelligence, Explanations, Reinforcement Learning

ABSTRACT :

We study reward decomposition for explain- ing the decisions of reinforcement learning (RL) agents. The approach decomposes rewards into sums of semantically meaningful reward types, so that actions can be compared in terms of trade-offs among the types. In particular, we introduce the concept of minimum sufficient ex- planations for compactly explaining why one action is preferred over another in terms of the types. Many prior RL algorithms for decom- posed rewards produced inconsistent decom- posed values, which can be ill-suited to expla- nation. We exploit an off-policy variant of Q- learning that provably converges to an optimal policy and the correct decomposed action val- ues. We illustrate the approach in a number of domains, showing its utility for explanations.
Authors :
Zoe Juozapaitis , Anurag Koul , Alan Fern , Martin Erwig , Finale Doshi-Velez

Paper Link :
http://web.engr.oregonstate.edu/~afern/papers/reward_decomposition__workshop_final.pdf

Venue :
2019 IJCAI/ECAI Workshop on Explainable Artificial Intelligence.

Related Works :

Sukkerd, Roykrong, Reid Simmons, and David Garlan. "Tradeoff-Focused Contrastive Explanation for MDP Planning." arXiv preprint arXiv:2004.12960 (2020).

https://arxiv.org/abs/2004.12960

Video : https://www.youtube.com/watch?v=3uIWTTyFtZs

Видео Explainable Reinforcement Learning via Reward Decomposition канала Papers and Chill