Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control
This work presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general control solution that can be used for a range of dynamic bipedal skills, from periodic walking and running to aperiodic jumping and standing.
Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot’s I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances.
The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.
Supplementary Videos:
- Running: https://youtu.be/ad3ZrvUzbXM
- 400m Dash: https://youtu.be/wzQtRaXjvAk
- Walking: https://youtu.be/vUewNUtSG3c
Видео Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control канала Hybrid Robotics
Our RL-based controller incorporates a novel dual-history architecture, utilizing both a long-term and short-term input/output (I/O) history of the robot. This control architecture, when trained through the proposed end-to-end RL approach, consistently outperforms other methods across a diverse range of skills in both simulation and the real world. The study also delves into the adaptivity and robustness introduced by the proposed RL system in developing locomotion controllers. We demonstrate that the proposed architecture can adapt to both time-invariant dynamics shifts and time-variant changes, such as contact events, by effectively using the robot’s I/O history. Additionally, we identify task randomization as another key source of robustness, fostering better task generalization and compliance to disturbances.
The resulting control policies can be successfully deployed on Cassie, a torque-controlled human-sized bipedal robot. This work pushes the limits of agility for bipedal robots through extensive real-world experiments. We demonstrate a diverse range of locomotion skills, including: robust standing, versatile walking, fast running with a demonstration of a 400-meter dash, and a diverse set of jumping skills, such as standing long jumps and high jumps.
Supplementary Videos:
- Running: https://youtu.be/ad3ZrvUzbXM
- 400m Dash: https://youtu.be/wzQtRaXjvAk
- Walking: https://youtu.be/vUewNUtSG3c
Видео Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control канала Hybrid Robotics
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Safety-Critical Geometric Control for Systems on ManifoldsRule-Based Safety-Critical Control Design using CBFs with Application to Autonomous Lane Change400m dash - RL for Versatile, Dynamic, and Robust Bipedal Locomotion ControlSupplementary Walking Experiments - RL for Versatile, Dynamic, and Robust Bipedal Locomotion ControlOptimal Robust Safety-Critical Control for Dynamic RoboticsHierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal RobotDynamic Legged Manipulation of a Ball through Multi-Contact OptimizationDynamic Walking on Stepping Stones with Gait Library and Control Barrier FunctionsGaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with UncertaintyDeep Visual Perception for Dynamic Walking on Discrete TerrainSupplementary Running Experiments - RL for Versatile, Dynamic, and Robust Bipedal Locomotion ControlDynamic Walking on Randomly-Varying Discrete Terrain with One-step PreviewDynamic Walking on Randomly-Varying Discrete Terrain with One-step PreviewCompetitive Car Racing with Multiple VehiclesSafe Teleoperation of Dynamic UAVs through Control Barrier FunctionsGeometric L1 Adaptive Attitude Control for Quadrotor UAVMotion Planning and Feedback Control for Bipedal Robots riding a Snakeboard3D Dynamic Walking on Stepping Stones with Control Barrier FunctionsRobust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement LearningDifferential Flatness based Direct Collocation for a Quadrotor with a Cable-Suspended Payload