PODNet: A Neural Network for Discovery of Plannable Options

Learning from demonstration has been widely studied in machine learning but becomes challenging when the demonstrated trajectories are unstructured and follow different objectives. PODNet or Plannable Option Discovery Network, addresses how to segment an unstructured set of demonstrated trajectories for option discovery. This enables learning from demonstration to perform multiple tasks and plan high-level trajectories based on the discovered option labels. PODNet combines a custom categorical variational autoencoder, a recurrent option inference network, option-conditioned policy network, and option dynamics model in an end-to-end learning architecture.

Behavior discovery for point robot in custom built CircleWorld environment.

The network had no prior idea about clockwise and counterclockwise motions and discovered two dynamically unqiue sub-behaviors by itself (inferred in red and blue) which happen to correspond well to human recognition of clockwise and counterclockwise motion as two sub-behaviors for this point robot.