Video Highlights
Robotics Challenge


Positions Available


Robot Locomotion Group




    The goal of our research is to build machines which exploit their natural dynamics to achieve extraordinary agility and efficiency. In an age where "big data" is all the rage, we still have relatively limited data from robots in these regimes, and instead rely mostly on existing models (e.g. from Lagrangian mechanics) and model-based optimization. We believe that deep connections are possible -- enabling very efficient optimization by exploiting structure in the governing equations -- and are working hard on both optimization algorithms and control applications. Our current projects include dynamics and control for humanoid robots, robotic manipulation, and dynamic walking over rough terrain, flight control for aggressive maneuvers in unmanned aerial vehicles, feedback control for fluid dynamics and soft robotics, and connections between perception and control.

    The Robot Locomotion Group is a part of Robotics @ MIT and CSAIL.

    Follow us on facebook and/or twitter.


Locomotion Group Paper and Multimedia News  

    A Pipeline for Generating Ground Truth Labels for Real {RGBD} Data of Cluttered Scenes
      by Pat Marion and Peter R. Florence and Lucas Manuelli and Russ Tedrake

      Deep neural network (DNN) architectures have been shown to outperform tradi- tional pipelines for object segmentation and pose estimation using RGBD data, but the performance of these DNN pipelines is directly tied to how representative the training data is of the true data. Hence a key requirement for employing these methods in practice is to have a large set of labeled data for your specific robotic manipulation task, a requirement that is not generally satisfied by existing datasets. In this paper we develop a pipeline to rapidly generate high quality RGBD data with pixelwise labels and object poses. We use an RGBD camera to collect video of a scene from multiple viewpoints and leverage existing reconstruction techniques to produce a 3D dense reconstruction. We label the 3D reconstruction using a human assisted ICP-fitting of object meshes. By reprojecting the results of labeling the 3D scene we can produce labels for each RGBD image of the scene. This pipeline enabled us to collect over 1,000,000 labeled object instances in just a few days. We use this dataset to answer questions related to how much training data is required, and of what quality the data must be, to achieve high performance from a DNN architecture.

      Supplemental materials: Marion17sup.pdf , Marion17sup2.mp4 , http://labelfusion.csail.mit.edu

      Under review. Comments welcome.

    Globally Optimal Object Pose Estimation in Point Clouds with Mixed-Integer Programming

      by Gregory Izatt and Russ Tedrake

      Motivated by the limitations of local object trackers, we present a formulation of the underlying point-cloud object pose estimation problem as a mixed-integer convex program, which we efficiently solve to optimality with an off-the-shelf branch and bound solver. We show that reasoning about object pose estimation in this way allows natural extension to point-to-mesh correspondence, multiple object pose estimation, and outlier rejection without losing the ability to obtain a globally optimal solution. We probe the extent to which rich problem-specific formulations typically tackled with unreliable nonlinear optimization can be rigorously treated in a global optimization framework to provide reliable initializations for powerful-but-local object trackers.

      Under review. Comments welcome.

    Global inverse kinematics via mixed-integer convex optimization

      by Hongkai Dai and Gregory Izatt and Russ Tedrake

      In this paper we present a novel formulation of the inverse kinematics (IK) problem with generic constraints as a mixed-integer convex optimization program. The proposed approach can solve the IK problem globally with generic task space constraints, a major improvement over existing approaches, which either solve the problem in only a local neighborhood of the user initial guess through nonlinear non-convex optimization, or address only a limited set of kinematics constraints. Specifically, we propose a mixed-integer convex relaxation on non-convex SO(3) rotation constraints, and apply this relaxation on the inverse kinematics problem. Our formulation can detect if an instance of the IK problem is globally infeasible, or produce an approximate solution when it is feasible. We show results on a 7- joint arm grasping objects in a cluttered environment, and a quadruped standing on stepping stones. We also compare our approach against the analytical approach for a 6-joint manipulator. The code is open-sourced at drake.mit.edu [29].

      Under review. Comments welcome.

    Tracking Objects with Point Clouds from Vision and Touch

      by Gregory Izatt and Geronimo Mirano and Edward Adelson and Russ Tedrake

      We present an object-tracking framework that fuses point cloud information from an RGB-D camera with tactile information from a GelSight contact sensor. GelSight can be treated as a source of dense local geometric information, which we incorporate directly into a conventional point-cloud-based articulated object tracker based on signed-distance functions. Our implementation runs at 12 Hz using an online depth reconstruction algorithm for GelSight and a modified second- order update for the tracking algorithm. We present data from hardware experiments demonstrating that the addition of contact-based geometric information significantly improves the pose accuracy during contact, and provides robustness to occlusions of small objects by the robotÂ’s end effector.

      Supplemental materials: https://www.youtube.com/watch?v=BIW_jq3dOEE

      Under review. Comments welcome.

    Planning robust walking motion on uneven terrain via convex optimization

      by Hongkai Dai and Russ Tedrake

      In this paper, we present a convex optimization problem to generate Center of Mass (CoM) and momentum trajectories of a walking robot, such that the motion robustly satisfies the friction cone constraints on uneven terrain. We adopt the Contact Wrench Cone (CWC) criterion to measure a robot's dynamical stability, which generalizes the venerable Zero Moment Point (ZMP) criterion. Unlike the ZMP criterion, which is ideal for walking on flat ground with unbounded tangential friction forces, the CWC criterion incorporates non-coplanar contacts with friction cone constraints. We measure the robustness of the motion using the margin in the Contact Wrench Cone at each time instance, which quantifies the capability of the robot to instantaneously resist external force/torque disturbance, without causing the foot to tip over or slide. For pre-specified footstep location and time, we formulate a convex optimization problem to search for robot linear and angular momenta that satisfy the CWC criterion. We aim to maximize the CWC margin to improve the robustness of the motion, and minimize the centroidal angular momentum (angular momentum about CoM) to make the motion natural. Instead of directly minimizing the non-convex centroidal angular momentum, we resort to minimizing a convex upper bound. We show that our CWC planner can generate motion similar to the result of the ZMP planner on flat ground with sufficient friction. Moreover, on an uneven terrain course with friction cone constraints, our CWC planner can still find feasible motion, while the outcome of the ZMP planner violates the friction limit.

      Under review. Comments welcome.


Locomotion Group News  

    May 19, 2017. Award. Pete Florence was awarded the EECS Masterworks award. Congratulations Pete!

    May 19, 2017. Award. Sarah Hensley was awarded the 2017 Best SuperUROP Presentation award. Congratulations Sarah!

    May 16, 2017. PhD Defense. Michael Posa successfully defended his thesis, titled "Optimization for Control and Planning of Multi-Contact Dynamic Motion". Congratulations Michael!

    May 15, 2017. Award. Our paper describing the planning and control that we implemented on Atlas for the DARPA Robotics Challenge was recognized with the IEEE-RAS Technical Commmittee on Whole-Body Control 2016 Best Paper of the Year award.

    January 28, 2017. Video. Amara Mesnik put together a great mini-documentary on MIT's entry in the DARPA Robotics Challenge.

    May 13, 2016. PhD Defense. Ani Majumdar has successfully defended his PhD thesis. Congratulations Ani! Click on the link to watch his talk, and check the publications page to read his thesis.

    February 24, 2016. Media. NOVA's documentary on the DARPA Robotics Challenge, titled "Rise of the Robots" is online now.

    December 7, 2015. PhD Defense. Andy Barry has successfully defended his PhD thesis. Congratulations Andy! Click on the link to watch his talk.

    November 18, 2015. In the news. NASA's R5 humanoid robot is coming to MIT. We're very excited to have the opportunity to do research on this amazing platform.

    November 5, 2015. Award. Our DRC Team's continuous walking with stereo fusion paper just won the Best Paper Award (Oral) at Humanoids 2015. Congratulations all!

    November 5, 2015. In the news. Andy's video of high-speed UAV obstacle avoidance (using only onboard processing) got some great coverage this week. This article by the IEEE Spectrum was particularly nice and insightful.

    October 26, 2015. PhD Defense. Andres Valenzuela just successfully defended his PhD thesis. Congratulations Andres!

    May 29, 2015. News. We're heading off to the DARPA Robotics Challenge. We've been posting some fun videos to our YouTube site (linked here). Wish us luck!

    May 26, 2015. News. Benoit Landry has submitted his Masters Thesis on Aggressive Quadrotor Flight in Dense Clutter. Be sure to check out his cool video.

    May 28, 2015. News. Scott Kuindersma has accepted a tenure-track position at Harvard starting this fall. Congratulations Scott!

    November 20, 2014. Award. Robin Deits' paper on Mixed-integer optimization for footstep planning just won the Best Paper Award (Oral) at Humanoids 2014. Congratulations Robin!

    December 31, 1969. Award. Benoit Landry has been awarded the 2014 Siebel Scholarship. Congratulations Benoit!

    June 18, 2014. News. Ram Vasudevan has officially accepted a tenure-track position at the University of Michigan, Ann Arbor. Congratulations Ram! Go Blue!

    June 16, 2014. News. Joe Moore has officially accepted a position at the Johns Hopkins Applied Phsyics Lab. Congratulations Joe!

    May 27, 2014. PhD Defense. Joseph Moore officially defended his PhD. You can watch his defense on the group talks page. Congratulations Joe!

For Group Members: