Best Paper Awards NeurIPS 2018: Non-delusional Q-learning and Value-iteration
Join the channel membership:
https://www.youtube.com/c/AIPursuit/join
Subscribe to the channel:
https://www.youtube.com/c/AIPursuit?sub_confirmation=1
Support and Donation:
Paypal ⇢ https://paypal.me/tayhengee
Patreon ⇢ https://www.patreon.com/hengee
BTC ⇢ bc1q2r7eymlf20576alvcmryn28tgrvxqw5r30cmpu
ETH ⇢ 0x58c4bD4244686F3b4e636EfeBD159258A5513744
Doge ⇢ DSGNbzuS1s6x81ZSbSHHV5uGDxJXePeyKy
-----------------------------------------------------------------------------------------
Credit to Tyler Lu and NeurIPS 2018
-----------------------------------------------------------------------------------------
The paper: https://papers.nips.cc/paper/8200-non-delusional-q-learning-and-value-iteration
The original conference video: https://www.facebook.com/nipsfoundation/videos/357066758186895/
Abstract:
We identify a fundamental source of error in Q-learning and other forms of dynamic programming with function approximation. Delusional bias arises when the approximation architecture limits the class of expressible greedy policies. Since standard Q-updates make globally uncoordinated action choices with respect to the expressible policy class, inconsistent or even conflicting Q-value estimates can result, leading to pathological behaviour such as over/under-estimation, instability and even divergence. To solve this problem, we introduce a new notion of policy consistency and define a local backup process that ensures global consistency through the use of information sets---sets that record constraints on policies consistent with backed-up Q-values. We prove that both the model-based and model-free algorithms using this backup remove delusional bias, yielding the first known algorithms that guarantee optimal results under general conditions. These algorithms furthermore only require polynomially many information sets (from a potentially exponential support). Finally, we suggest other practical heuristics for value-iteration and Q-learning that attempt to reduce delusional bias.
Q-Learning, Deep Reinforcement Learning
Видео Best Paper Awards NeurIPS 2018: Non-delusional Q-learning and Value-iteration канала AI Pursuit by TAIR
https://www.youtube.com/c/AIPursuit/join
Subscribe to the channel:
https://www.youtube.com/c/AIPursuit?sub_confirmation=1
Support and Donation:
Paypal ⇢ https://paypal.me/tayhengee
Patreon ⇢ https://www.patreon.com/hengee
BTC ⇢ bc1q2r7eymlf20576alvcmryn28tgrvxqw5r30cmpu
ETH ⇢ 0x58c4bD4244686F3b4e636EfeBD159258A5513744
Doge ⇢ DSGNbzuS1s6x81ZSbSHHV5uGDxJXePeyKy
-----------------------------------------------------------------------------------------
Credit to Tyler Lu and NeurIPS 2018
-----------------------------------------------------------------------------------------
The paper: https://papers.nips.cc/paper/8200-non-delusional-q-learning-and-value-iteration
The original conference video: https://www.facebook.com/nipsfoundation/videos/357066758186895/
Abstract:
We identify a fundamental source of error in Q-learning and other forms of dynamic programming with function approximation. Delusional bias arises when the approximation architecture limits the class of expressible greedy policies. Since standard Q-updates make globally uncoordinated action choices with respect to the expressible policy class, inconsistent or even conflicting Q-value estimates can result, leading to pathological behaviour such as over/under-estimation, instability and even divergence. To solve this problem, we introduce a new notion of policy consistency and define a local backup process that ensures global consistency through the use of information sets---sets that record constraints on policies consistent with backed-up Q-values. We prove that both the model-based and model-free algorithms using this backup remove delusional bias, yielding the first known algorithms that guarantee optimal results under general conditions. These algorithms furthermore only require polynomially many information sets (from a potentially exponential support). Finally, we suggest other practical heuristics for value-iteration and Q-learning that attempt to reduce delusional bias.
Q-Learning, Deep Reinforcement Learning
Видео Best Paper Awards NeurIPS 2018: Non-delusional Q-learning and Value-iteration канала AI Pursuit by TAIR
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Follow up Neural ODEs @ NeurIPS 2019 (NeurIPS 2018 Best Paper)
Knowledge Enhanced Contextual Word Representations | EMNLP-IJCNLP2019
NYU Deep Learning Week 14 – Lecture: Structured prediction with energy-based models | Yann LeCun
Yoshua Bengio | Meta-transfer Learning | ICLR 2019
Continuous Optimization for Structure Learning | NeurIPS 2018
Reinforcement Learning: Past, Present, and Future Perspectives (w/ slides) | NeurIPS 2019
3D | Deep Learning for 3D Vision
Graph Neural Network | Tutorial on Graph-based Deep Learning in NLP | Part 2
ICLR 2020 | Yoshua Bengio, Yann LeCun Reflections (Self-Supervised Learning)
NLP | Text Data Augmentation with PyTorch datasets
Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus
NeurIPS 2019 Track 1 Session 4
Yoshua Bengio: Deep Learning Priors Associated With Conscious Processing | ICLR 2020
Causal Inference and Stable Learning by Peng Cui
Convolutional Neural Networks (CNN) What & How it works - Brandon Rohrer
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding | NAACL BEST PAPERS
Deep Learning with Label Noise | Kevin McGuinness
NYU Deep Learning Week 13 – Practicum: Graph Convolutional Networks (GCNs)
Graph Neural Network | Tutorial on Benchmarking GNN by Xavier Bresson & Yoshua Bengio
fastai v2 | Deep Learning for Coders: Lesson 1 | Jeremy Howard | Rachel Thomas
ICLR 2020 | Q & A w/ Yoshua Bengio, Yann LeCun Reflections ft. Dawn Song