AI60209: AI/ML for Robot Autonomy

Objective

This course presents AI/ML based principles for endowing robots with capabilities to autonomously learn new skills to make the control of robots safe. The global robotics technology market size is expected to surpass around 283 billion USD by 2032 (Ref). The aim of this course is to introduce students with basics as well as modern robot autonomy. Students will be able to understand why AI/ML based methods are required in the modern robotics industry and why the traditional AI/ML is not directly applicable. Students will be introduced to the following

  • a set of control laws that enable motion- planning by learning the dynamical systems
  • reinforcement learning and its relationship to optimal control for robotics, and imitation learning

Modus Operandi

  • Class-room activities: 10%
    • Participate in class-room discussions and ask quality questions
    • Five minutes teaching: Pick any small topic that is taught in the class before your schedule and teach it for five minutes.
    • Attendance
  • Course project: 20%
    • I will provide you codes. Your job will be to run that code by yourself and understand the algorithms and associated theory.
  • Mid-sem: 30%
  • End-sem: 40%

Syllabus

Module 1: Foundation for robotics and control

Robot body (introduction and rigid transformation), Robot motion (SO3, screw, twist and inverse kinematics), Robot dynamics (angular momentum, rotational inertia, kinetic energy, force, change of frames), cart-pole, quadrotors (simulation using ROS/ Gazebo), mobile robots (simulation using ROS/ Gazebo), Open-loop vs. closed-loop control, notions of safety, chance constraints, stability, stabilizability, controllability and reachability

Module 2: Autonomy with dynamical systems

Learning from demonstration: teaching robots via human demonstration such as teleoperation, kinesthetic teaching and observational learning

Learning a control law: three classic regression methods to estimate model and demonstrates their inability to learn a stable dynamical system (DS), brief introduction to Lyapunov theory, Gaussian mixture regression, physically consistent estimation approach for Gaussian mixture models

Adapting and modulating an existing control law: modulations that act locally to preserve the generic properties of the nominal DS (e.g., asymptotic or global stability)

Module 3: Autonomy under constraints

Obstacle avoidance: modulate the DS to contour obstacles or to remain within a given workspace, Model Predictive Control (MPC), Deep MPC, stochastic MPC and constrained estimator

Module 4: Autonomy with RL

Robots as Markov Decision Problems, Intro to RL: Sequential decision making examples of robotics, principle of optimality, dynamic programming, examples of uncertainty in the robotics, challenges and extensions of dynamic programming

Model-based and model-free RL for robot control: problem formulation, value iteration, policy iteration, Q learning, policy gradient, actor-critic, deep RL for robotics

Deep RL: DQN (Deep Q learning, deep value network, training deep Q network), Unbiased policy gradient method

Advanced on-policy RL and off-policy RL: practical first order policy optimization, efficient and stable policy optimization, incremental Monte Carlo value function estimation, trust region method, deep deterministic policy gradient, trouble and tricks in robotics, soft actor critic

Imitation learning: Behavioral cloning, direct policy learning, inverse RL, learning from comparison and physical feedback

Resources

  1. A Mathematical Introduction to Robotic Manipulation by Richard Murray, Zexiang Li and Shankar Sastry, CRC Press
  2. Learning for Adaptive and Reactive Robot Control By Aude Billard, Sina Mirrazavi and Nadia Figueroa, MIT press
  3. Lecture notes of Machine Learning for Robotics by Hao Su
  4. Lecture notes of Principles of Robot Autonomy II by Marco Pavone, Dorsa Sadigh and Jeannette Bohg
  5. Springer Handbook of Robotics, Editors: B. Siciliano and O. Khatib
  6. Robotics, Vision and Control by Peter Corke
  7. Probabilistic Robotics by Sebastian Thrun, Wolfram Burgard and Dieter Fox
  8. Lecture notes on Underactuated Robotics by Russ Tedrake
  9. Lecture notes of Robot Learning by Sanjiban Choudhury