Video Highlights
Robotics Challenge


Positions Available


Robot Locomotion Group




    The goal of our research is to build machines which exploit their natural dynamics to achieve extraordinary agility and efficiency. In an age where "big data" is all the rage, we still have relatively limited data from robots in these regimes, and instead rely mostly on existing models (e.g. from Lagrangian mechanics) and model-based optimization. We believe that deep connections are possible -- enabling very efficient optimization by exploiting structure in the governing equations -- and are working hard on both optimization algorithms and control applications. Our previous projects have included dynamics and control for humanoid robots, dynamic walking over rough terrain, flight control for aggressive maneuvers in unmanned aerial vehicles, feedback control for fluid dynamics and soft robotics, and connections between perception and control. These days the lab is primarily focused on robot manipulation, with continued emphasis on feedback control (which is so far largely absent in manipulation) and the connections between perception and control.

    The Robot Locomotion Group is a part of Robotics @ MIT and CSAIL.

    Follow us on facebook and/or twitter.


Locomotion Group Paper and Multimedia News  

    Warm Start of Mixed-Integer Programs for Model Predictive Control of Hybrid Systems
      by Tobia Marcucci and Russ Tedrake

      Model Predictive Control (MPC) has revolutionized the standards for control of smooth dynamical systems, allowing the synthesis of reliable optimal controllers which can naturally cope with constraints, and whose intuitive parameterization greatly simplifies control design. However, the discrete nature of hybrid systems, which lends itself naturally to transcription as a mixed-integer optimization problem, results in programs that are generally considered to be too computationally expensive to be used in real-time control applications. Although the branch-and-bound algorithm has a worst-case cost that is exponential in the number of integer variables, in MPC we expect to solve a nearly identical optimization problem on the very next time step. Surprisingly, the concept of warm-starting the solver (by reusing the computation from the previous time step), that has had great success in optimal control of linear systems, has not yet been used effectively in mixed-integer MPC formulations. In this paper we present a novel warm-start technique for hybrid MPC that uses dual solutions from one time step to imply rigorous lower bounds on the solution tree for the next time step, even in the presence of disturbances, allowing branch-and-bound to prune very quickly. In practice, we find that this method often reduces computation to the combinatoric complexity of only a one-step lookahead problem.

      Under review. Comments welcome.

    {kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation}

      by Lucas Manuelli* and Wei Gao* and Peter Florence and Russ Tedrake

      We would like robots to achieve purposeful manipulation by placing any instance from a category of objects into a desired set of goal states. Existing manipulation pipelines typically specify the desired configuration as a target 6-DOF pose and rely on explicitly estimating the pose of the manipulated objects. However, representing an object with a parameterized transformation defined on a fixed template cannot capture large intra-category shape variation, and specifying a target pose at a category level can be physically infeasible or fail to accomplish the task -- e.g. knowing the pose and size of a coffee mug relative to some canonical mug is not sufficient to successfully hang it on a rack by its handle. Hence we propose a novel formulation of category-level manipulation that uses semantic 3D keypoints as the object representation. This keypoint representation enables a simple and interpretable specification of the manipulation target as geometric costs and constraints on the keypoints, which flexibly generalizes existing pose-based manipulation methods. Using this formulation, we factor the manipulation policy into instance segmentation, 3D keypoint detection, optimization-based robot action planning and local dense-geometry-based action execution. This factorization allows us to leverage advances in these sub-problems and combine them into a general and effective perception-to-action manipulation pipeline. Our pipeline is robust to large intra-category shape variation and topology changes as the keypoint representation ignores task-irrelevant geometric details. Extensive hardware experiments demonstrate our method can reliably accomplish tasks with never-before seen objects in a category, such as placing shoes and mugs with significant shape variation into category level target configurations.

      Under review. Comments welcome.

    Linear Encodings for Polytope Containment Problems

      by Sadra Sadraddini and Russ Tedrake

      The polytope containment problem is deciding whether a polytope is a contained within another polytope. This problem is rooted in computational convexity, and arises in applications such as verification and control of dynamical systems. The complexity heavily depends on how the polytopes are represented. Describing polytopes by their hyperplanes (H-polytopes) is a popular representation. In many applications we use affine transformations of H-polytopes, which we refer to as AH-polytopes. Zonotopes, orthogonal projections of H-polytopes, and convex hulls/Minkowski sums of multiple H-polytopes can be efficiently represented as AH-polytopes. While there exists efficient necessary and sufficient conditions for AH-polytope in H-polytope containment, the case of AH-polytope in AH-polytope is known to be NP-complete. In this paper, we provide a sufficient condition for this problem that is cast as a linear program with size that grows linearly with the number of hyperplanes of each polytope. Special cases on zonotopes, Minkowski sums, convex hulls, and disjunctions of H-polytopes are studied. These efficient encodings enable us to designate certain components of polytopes as decision variables, and incorporate them into a convex optimization problem. We present examples on the zonotope containment problem, polytopic Hausdorff distances, zonotope order reduction, inner approximations of orthogonal projections, and demonstrate the usefulness of our results on formal controller verification and synthesis for hybrid systems.

      Under review. Comments welcome.

    Mixed-Integer Formulations for Optimal Control of Piecewise-Affine Systems

      by Tobia Marcucci and Russ Tedrake

      In this paper we study how to formulate the optimal control problem for a piecewise-affine dynamical system as a mixed-integer program. Problems of this form arise typically in hybrid Model Predictive Control (MPC), where at every time step an open-loop optimal control sequence is computed via numerical optimization and applied to the system in a moving horizon fashion. Not surprisingly, the efficiency in the formulation of the underlying mathematical program has a crucial influence on computation times, and hence on the applicability of hybrid MPC to high-dimensional systems. We leverage on modern concepts and results from the fields of mixed-integer and disjunctive programming to conduct a comprehensive analysis of this formulation problem: among the outcomes enabled by this novel perspective is the derivation of multiple highly-efficient formulations of the control problem, each of which represents a different tradeoff between the two most important features of a mixed-integer program, the size and the strength. First in theory, then through a numerical example, we show how all the proposed methods outperform the traditional approach employed in MPC, enabling the solution of larger-scale problems.

      Under review. Comments welcome.

    Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids

      by Yunzhu Li and Jiajun Wu and Russ Tedrake and Joshua B. Tenenbaum and Antonio Torralba

      Real-life control tasks involve matters of various substances---rigid or soft bodies, liquid, gas---each with distinct physical behaviors. This poses challenges to traditional rigid-body physics engines. Particle-based simulators have been developed to model the dynamics of these complex scenes; however, relying on approximation techniques, their simulation often deviates from real-world physics, especially in the long term. In this paper, we propose to learn a particle-based simulator for complex control tasks. Combining learning with particle-based systems brings in two major benefits: first, the learned simulator, just like other particle-based systems, acts widely on objects of different materials; second, the particle-based representation poses strong inductive bias for learning: particles of the same type have the same dynamics within. This enables the model to quickly adapt to new environments of unknown dynamics within a few observations. We demonstrate robots achieving complex manipulation tasks using the learned simulator, such as manipulating fluids and deformable foam, with experiments both in simulation and in the real world. Our study helps lay the foundation for robot learning of dynamic scenes with particle-based representations.

      Supplemental materials: http://dpi.csail.mit.edu/ , https://www.youtube.com/watch?v=nQipmVDuytQ , https://arxiv.org/abs/1810.01566

      Under review. Comments welcome.


Locomotion Group News  

    October 15, 2018. PhD Defense. Congratulations to Robin Deits for successfully defending his thesis!

    October 3, 2018. Award. Congratulations to Pete Florence and Lucas Manuelli whose paper Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation won the Conference Best Paper Award at CoRL 2018!

    September 19, 2018. Award. Congratulations to Pete Florence and Lucas Manuelli whose paper Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation won the first ever Amazon Robotics Best Technical Paper Award (2018).

    June 18, 2018. Award. Congratulations to Ani Majumdar whose paper Funnel libraries for real-time robust feedback motion planning won the first ever International Journal of Robotics Research Paper of the Year (2017).

    April 26, 2018. Award. Congratulations to Katy Muhlrad for winning the "Audience Choice Award" at the SuperUROP Showcase for her work on "Using GelSight to Identify Objects by Touch".

    July 26, 2017. Defense. Frank Permenter successfully defended his thesis, titled "Reduction methods in semidefinite and conic optimization". Congratulations Frank!

    May 19, 2017. Award. Pete Florence was awarded the EECS Masterworks award. Congratulations Pete!

    May 19, 2017. Award. Sarah Hensley was awarded the 2017 Best SuperUROP Presentation award. Congratulations Sarah!

    May 16, 2017. PhD Defense. Michael Posa successfully defended his thesis, titled "Optimization for Control and Planning of Multi-Contact Dynamic Motion". Congratulations Michael!

    May 15, 2017. Award. Our paper describing the planning and control that we implemented on Atlas for the DARPA Robotics Challenge was recognized with the IEEE-RAS Technical Commmittee on Whole-Body Control 2016 Best Paper of the Year award.

    January 28, 2017. Video. Amara Mesnik put together a great mini-documentary on MIT's entry in the DARPA Robotics Challenge.

    May 13, 2016. PhD Defense. Ani Majumdar has successfully defended his PhD thesis. Congratulations Ani! Click on the link to watch his talk, and check the publications page to read his thesis.

    February 24, 2016. Media. NOVA's documentary on the DARPA Robotics Challenge, titled "Rise of the Robots" is online now.

    December 7, 2015. PhD Defense. Andy Barry has successfully defended his PhD thesis. Congratulations Andy! Click on the link to watch his talk.

    November 18, 2015. In the news. NASA's R5 humanoid robot is coming to MIT. We're very excited to have the opportunity to do research on this amazing platform.

    November 5, 2015. Award. Our DRC Team's continuous walking with stereo fusion paper just won the Best Paper Award (Oral) at Humanoids 2015. Congratulations all!

    November 5, 2015. In the news. Andy's video of high-speed UAV obstacle avoidance (using only onboard processing) got some great coverage this week. This article by the IEEE Spectrum was particularly nice and insightful.

    October 26, 2015. PhD Defense. Andres Valenzuela just successfully defended his PhD thesis. Congratulations Andres!

    May 29, 2015. News. We're heading off to the DARPA Robotics Challenge. We've been posting some fun videos to our YouTube site (linked here). Wish us luck!

    May 26, 2015. News. Benoit Landry has submitted his Masters Thesis on Aggressive Quadrotor Flight in Dense Clutter. Be sure to check out his cool video.

For Group Members: