MIT Autonomous Vehicle: Learning Robust Sim-to-Real Control Policies
Learning Robust Control Policies for End-to-End Autonomous Driving From Data-Driven Simulation
(The VISTA Simulation Engine)
In this work, we present a data-driven simulation and training engine capable of learning end-to-end autonomous vehicle control policies using only sparse rewards. By leveraging real, human-collected trajectories through an environment, we render novel training data that allows virtual agents to drive along a continuum of new local trajectories consistent with the road appearance and semantics, each with a different view of the scene. We demonstrate the ability of policies learned within our simulator to generalize to and navigate in previously unseen real-world roads, without access to any human control labels during training. Our results validate the learned policy onboard a full-scale autonomous vehicle, including in previously un-encountered scenarios, such as new roads and novel, complex, near-crash situations. Our methods are scalable, leverage reinforcement learning, and apply broadly to situations requiring effective perception and robust operation in the physical world.
This paper was published in IEEE Robotics and Automation Letters and will be presented at ICRA 2020.
Technical Paper: https://ieeexplore.ieee.org/document/8957584
Code: coming soon!! https://forms.gle/4CzEou4Ffs9Lg9dr7
Project website: http://bit.ly/VISTA-sim
Authors: Alexander Amini, Igor Gilitschenski, Jacob Phillips, Julia Moseyko, Rohan Banerjee, Sertac Karaman, Daniela Rus
Acknowledgments:
Support for this work was given by the National Science Foundation (NSF) and Toyota Research Institute (TRI). However, note that this article solely reflects the opinions and conclusions of its authors and not TRI or any other Toyota entity. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the V100 GPU and Drive PX2 used for this research.
Видео MIT Autonomous Vehicle: Learning Robust Sim-to-Real Control Policies канала Alexander Amini
(The VISTA Simulation Engine)
In this work, we present a data-driven simulation and training engine capable of learning end-to-end autonomous vehicle control policies using only sparse rewards. By leveraging real, human-collected trajectories through an environment, we render novel training data that allows virtual agents to drive along a continuum of new local trajectories consistent with the road appearance and semantics, each with a different view of the scene. We demonstrate the ability of policies learned within our simulator to generalize to and navigate in previously unseen real-world roads, without access to any human control labels during training. Our results validate the learned policy onboard a full-scale autonomous vehicle, including in previously un-encountered scenarios, such as new roads and novel, complex, near-crash situations. Our methods are scalable, leverage reinforcement learning, and apply broadly to situations requiring effective perception and robust operation in the physical world.
This paper was published in IEEE Robotics and Automation Letters and will be presented at ICRA 2020.
Technical Paper: https://ieeexplore.ieee.org/document/8957584
Code: coming soon!! https://forms.gle/4CzEou4Ffs9Lg9dr7
Project website: http://bit.ly/VISTA-sim
Authors: Alexander Amini, Igor Gilitschenski, Jacob Phillips, Julia Moseyko, Rohan Banerjee, Sertac Karaman, Daniela Rus
Acknowledgments:
Support for this work was given by the National Science Foundation (NSF) and Toyota Research Institute (TRI). However, note that this article solely reflects the opinions and conclusions of its authors and not TRI or any other Toyota entity. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the V100 GPU and Drive PX2 used for this research.
Видео MIT Autonomous Vehicle: Learning Robust Sim-to-Real Control Policies канала Alexander Amini
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Alexander Amini - Nationwide TVAlexander Amini Explaining His Project During EUCYS 2011MIT 6.S191: Taming Dataset Bias via Domain AdaptationMIT Deep Learning 6.S191 TeaserMIT 6.S191: Deep Generative ModelingMIT Introduction to Deep Learning | 6.S191LiDAR-Based End-to-End Navigation | ICRA 2021MIT 6.S191 (2023): The Future of Robot LearningMIT 6.S191 (2021): Deep Learning New FrontiersMIT Introduction to Deep Learning (2023) | 6.S191MIT 6.S191 (2020): Convolutional Neural NetworksLearning Autonomous Driving in SimulationMIT 6.S191: Evidential Deep Learning and UncertaintyMIT 6.S191 (2021): Convolutional Neural NetworksMIT 6.S191 (2021): Introduction to Deep LearningMIT 6.S191: Convolutional Neural NetworksMIT 6.S191: Recurrent Neural Networks, Transformers, and AttentionMIT 6.S191 (2023): Convolutional Neural NetworksMIT 6.S191 (2020): Reinforcement LearningMIT 6.S191 (2020): Introduction to Deep Learning