Video Highlights
Robotics Challenge


Positions Available


Robot Locomotion Group




    The goal of our research is to build machines which exploit their natural dynamics to achieve extraordinary agility and efficiency. In an age where "big data" is all the rage, we still have relatively limited data from robots in these regimes, and instead rely mostly on existing models (e.g. from Lagrangian mechanics) and model-based optimization. We believe that deep connections are possible -- enabling very efficient optimization by exploiting structure in the governing equations -- and are working hard on both optimization algorithms and control applications. Our previous projects have included dynamics and control for humanoid robots, dynamic walking over rough terrain, flight control for aggressive maneuvers in unmanned aerial vehicles, feedback control for fluid dynamics and soft robotics, and connections between perception and control. These days the lab is primarily focused on robot manipulation, with continued emphasis on feedback control (which is so far largely absent in manipulation) and the connections between perception and control.

    The Robot Locomotion Group is a part of Robotics @ MIT and CSAIL.

    Follow us on facebook and/or twitter.


Locomotion Group Paper and Multimedia News  

    Warm Start of Mixed-Integer Programs for Model Predictive Control of Hybrid Systems
      by Tobia Marcucci and Russ Tedrake

      In hybrid Model Predictive Control (MPC), a Mixed-Integer Convex Program (MICP) is solved at each sampling time to compute the optimal control action. Although these optimizations are generally very demanding, in MPC we expect consecutive problem instances to be nearly identical. This paper addresses the question of how computations performed at one time step can be reused to accelerate ('warm start') the solution of subsequent MICPs. Reoptimization is not a rare practice in integer programming: for small variations of certain problem data, the branch-and-bound algorithm allows an efficient reuse of its search tree and the dual bounds of its leaf nodes. In this paper we extend these ideas to the receding-horizon settings of MPC. The warm-start algorithm we propose copes naturally with arbitrary model errors, has negligible computational cost, and frequently enables an a-priori pruning of most of the search space. Theoretical considerations and experimental evidence show that the proposed method tends to reduce the combinatorial complexity of the hybrid MPC problem to that of a one-step look-ahead optimization, greatly easing the online computation burden.

      Supplemental materials: http://arxiv.org/abs/1910.08251

      Under review (arXiv available). Comments welcome.

    The Nearest Polytope Problem: Algorithms and Application to Controlling Hybrid Systems

      by Albert Wu and Sadra Sadraddini and Russ Tedrake

      We present three algorithms for solving the nearest polytope problem: given a list of polytopes and a distance metric in Euclidean space, find the nearest polytope to a query point. We consider the AH-polytope representation, which generalizes the H-polytope representation and is particularly useful in the context of control. Through preprocessing the polytopes into efficient data structures, we avoid exhaustive search at query time. We discuss the properties of the algorithms and compare their performances using polytopic datasets motivated by control applications, including sampling-based motion planning and model predictive control.

      Under review. Comments welcome.

    Self-Supervised Correspondence in Visuomotor Policy Learning

      by Peter Florence and Lucas Manuelli and Russ Tedrake

      In this paper we explore using self-supervised correspondence for improving the generalization performance and sample efficiency of visuomotor policy learning. Prior work has primarily used approaches such as autoencoding, pose-based losses, and end-to-end policy optimization in order to train the visual portion of visuomotor policies. We instead propose an approach using self-supervised dense visual correspondence training, and show this enables visuomotor policy learning with surprisingly high generalization performance with modest amounts of data: using imitation learning, we demonstrate extensive hardware validation on challenging manipulation tasks with as few as 50 demonstrations. Our learned policies can generalize across classes of objects, react to deformable object configurations, and manipulate textureless symmetrical objects in a variety of backgrounds, all with closed-loop, real-time vision-based policies. Simulated imitation learning experiments suggest that correspondence training offers sample complexity and generalization benefits compared to autoencoding and end-to-end training.

      Supplemental materials: https://sites.google.com/view/visuomotor-correspondence , https://youtu.be/nDRBKb4AGmA

      Under review. Comments welcome.

    Generative Modeling of Environments with Scene Grammars and Variational Inference

      by Gregory Izatt and Russ Tedrake

      How do we verify that a cleaning robot that we have tested only in a simulator and in case studies in the lab, will work in every house in the world? A critical step in answering that question is to establish a quantitative understanding of the distribution of environments that a robot will face when when it is deployed. However, even restricting attention only to the distribution of objects in a scene, these distributions over environments are nontrivial: they describe mixtures of discrete and continuous variables related to the number, type, poses, and attributes of objects in the scene. We describe a probabilistic generative model that uses scene trees to capture hierarchical relationships between collections of objects, as well as a variational inference algorithm for tuning that model to best match a set of observed environments without any need for tediously labeled parse trees. We demonstrate that this model can accurately capture the distribution of a pair of nontrivial manipulation-relevant datasets and be deployed as a density estimator and outlier detector for novel environments.

      Under review. Comments welcome.

    R3T: Rapidly-exploring Random Reachable Set Tree for Optimal Kinodynamic Planning of Nonlinear Hybrid Systems

      by Albert Wu and Sadra Sadraddini and Russ Tedrake

      We introduce R3T, a reachability-based variant of the rapidly-exploring random tree (RRT) algorithm that is suitable for (optimal) kinodynamic planning in nonlinear and hybrid systems. We developed tools to approximate reachable sets using polytopes and perform sampling-based planning with them. This method provides a unique advantage in the case of hybrid systems: different dynamic modes in the reachable set can be explicitly represented using multiple polytopes. We show that R3T retains probabilistic completeness and asymptotic optimality of RRT/RRT*. Moreover, R3T provides a formal verification method for reachability analysis of nonlinear systems. The advantages of R3T are demonstrated with case studies on nonlinear, hybrid, and contact-rich robotic systems.

      Supplemental materials: https://youtu.be/E8TICePNqE0

      Under review. Comments welcome.


Locomotion Group News  

    September 18, 2019. PhD Defense. Congratulations to Twan Koolen for successfully defending his thesis!

    August 19, 2019. PhD Defense. Congratulations to Pete Florence for successfully defending his thesis!

    October 15, 2018. PhD Defense. Congratulations to Robin Deits for successfully defending his thesis!

    October 3, 2018. Award. Congratulations to Pete Florence and Lucas Manuelli whose paper Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation won the Conference Best Paper Award at CoRL 2018!

    September 19, 2018. Award. Congratulations to Pete Florence and Lucas Manuelli whose paper Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation won the first ever Amazon Robotics Best Technical Paper Award (2018).

    June 18, 2018. Award. Congratulations to Ani Majumdar whose paper Funnel libraries for real-time robust feedback motion planning won the first ever International Journal of Robotics Research Paper of the Year (2017).

    April 26, 2018. Award. Congratulations to Katy Muhlrad for winning the "Audience Choice Award" at the SuperUROP Showcase for her work on "Using GelSight to Identify Objects by Touch".

    July 26, 2017. Defense. Frank Permenter successfully defended his thesis, titled "Reduction methods in semidefinite and conic optimization". Congratulations Frank!

    May 19, 2017. Award. Pete Florence was awarded the EECS Masterworks award. Congratulations Pete!

    May 19, 2017. Award. Sarah Hensley was awarded the 2017 Best SuperUROP Presentation award. Congratulations Sarah!

    May 16, 2017. PhD Defense. Michael Posa successfully defended his thesis, titled "Optimization for Control and Planning of Multi-Contact Dynamic Motion". Congratulations Michael!

    May 15, 2017. Award. Our paper describing the planning and control that we implemented on Atlas for the DARPA Robotics Challenge was recognized with the IEEE-RAS Technical Commmittee on Whole-Body Control 2016 Best Paper of the Year award.

    January 28, 2017. Video. Amara Mesnik put together a great mini-documentary on MIT's entry in the DARPA Robotics Challenge.

    May 13, 2016. PhD Defense. Ani Majumdar has successfully defended his PhD thesis. Congratulations Ani! Click on the link to watch his talk, and check the publications page to read his thesis.

    February 24, 2016. Media. NOVA's documentary on the DARPA Robotics Challenge, titled "Rise of the Robots" is online now.

    December 7, 2015. PhD Defense. Andy Barry has successfully defended his PhD thesis. Congratulations Andy! Click on the link to watch his talk.

    November 18, 2015. In the news. NASA's R5 humanoid robot is coming to MIT. We're very excited to have the opportunity to do research on this amazing platform.

    November 5, 2015. Award. Our DRC Team's continuous walking with stereo fusion paper just won the Best Paper Award (Oral) at Humanoids 2015. Congratulations all!

    November 5, 2015. In the news. Andy's video of high-speed UAV obstacle avoidance (using only onboard processing) got some great coverage this week. This article by the IEEE Spectrum was particularly nice and insightful.

    October 26, 2015. PhD Defense. Andres Valenzuela just successfully defended his PhD thesis. Congratulations Andres!

For Group Members: