Visual Reaction

Visual reaction: the task of interacting with dynamic environments where
the changes in the environment are not necessarily caused by the agents
itself. Visual reaction entails predicting the future changes in a
visual environment and planning accordingly. We study the problem of
visual reaction in the context of playing catch with a drone in visually
rich synthetic environments. This is a challenging problem since the
agent is required to learn (1) how objects with different physical
properties and shapes move, (2) what sequence of actions should be taken
according to the prediction, (3) how to adjust the actions based on the
visual feedback from the dynamic environment (e.g., when objects
bouncing off a wall), and (4) how to reason and act with an unexpected
state change in a timely manner.

Видео Visual Reaction канала Allen Institute for AI