Home
People
Publications
Talks/Lectures
Video Highlights
Robotics Challenge
Software
Directions
Accessibility
Robot Locomotion Group
The goal of our research is to build machines which exploit their natural dynamics to achieve extraordinary agility, efficiency, and robustness using rigorous tools from dynamical systems, control theory, and machine learning. Our current focus in on robotic manipulation, because the revolution in recent machine learning has opened a pathway in these applications to merging control theory and perception at a level that has never been considered before; ideas like "intuitive physics" and "common-sense reasoning" will meet with rigorous ideas like "model-order reduction" and "robust/adaptive control". It's going to be a great few years!
Our previous projects have included dynamics and control for humanoid robots, dynamic walking over rough terrain, flight control for aggressive maneuvers in unmanned aerial vehicles, feedback control for fluid dynamics and soft robotics, and connections between perception and control.
The Robot Locomotion Group is a part of Robotics @ MIT and CSAIL.
Locomotion Group Paper and Multimedia News 
-
Graphs of Convex Sets with Applications to Optimal Control and Motion Planning
- by Tobia Marcucci
This thesis introduces a new class of problems at the interface of combinatorial and convex optimization. We consider graphs where each vertex is paired with a convex program, and each edge couples two programs through additional convex costs and constraints. We call such a graph a Graph of Convex Sets (GCS). Over a GCS we can formulate any optimization problem that we can formulate over an ordinary weighted graph, with scalar costs on the vertices and edges. In fact, for any fixed choice of the variables in the convex programs, a GCS reduces to a weighted graph where we can seek, e.g., a path, a matching, a tour, or a spanning tree of minimum cost. The challenge in a GCS problem lies in solving the discrete and the continuous components of the problem jointly. By combining the modelling power of graphs and convex optimization, GCSs are a flexible framework to formulate and solve many real-world problems. The graph and the combinatorial goal (e.g., finding a path or a tour) model the high-level discrete skeleton of a problem. The convex costs and constraints fill in the low-level continuous details. The primary contribution of this thesis is an efficient and unified method for solving any GCS problem. Starting from an integer-linear-programming formulation of an optimization problem over a weighted graph, this method formulates the corresponding GCS problem as an efficient Mixed-Integer Convex Program (MICP). This MICP can then be solved to global optimality using common branch-and-bound solvers, or approximately by rounding the solution of its convex relaxation. Importantly, both the formulation of the MICP and its solution are fully automatic, and a user of our framework does not need any expertise in mixed-integer optimization. We first describe the GCS framework and the formulation of our MICP in general terms, without presupposing the specific combinatorial problem to be solved over the GCS. We illustrate our techniques through multiple examples spanning logistics, transportation, scheduling, navigation, and computational geometry. Then we focus on the Shortest-Path Problem (SPP) in GCS. This problem is particularly interesting since it generalizes a wide variety of multi-stage decision-making problems and, using our techniques, it can be solved very effectively. We consider two main applications of the SPP in GCS: optimal control of dynamical systems and collision-free motion planning. In these two areas, our techniques either generalize or significantly improve upon algorithms and optimization methods that have been developed for decades and are widely used in academia and industry. Lastly, the techniques introduced in this thesis are implemented in the software packages Drake and gcspy. The former is a large and mature software for robotics. It is open-source and widely used by the community. The second is a very simple and lightweight Python package which is also open source. In this thesis, we will illustrate the usage of gcspy through multiple basic examples.
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
- by Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song
This paper introduces Diffusion Policy, a new way of generating robot behavior by representing a robot's visuomotor policy as a conditional denoising diffusion process. We benchmark Diffusion Policy across 12 different tasks from 4 different robot manipulation benchmarks and find that it consistently outperforms existing state-of-the-art robot learning methods with an average improvement of 46.9%. Diffusion Policy learns the gradient of the action-distribution score function and iteratively optimizes with respect to this gradient field during inference via a series of stochastic Langevin dynamics steps. We find that the diffusion formulation yields powerful advantages when used for robot policies, including gracefully handling multimodal action distributions, being suitable for high-dimensional action spaces, and exhibiting impressive training stability. To fully unlock the potential of diffusion models for visuomotor policy learning on physical robots, this paper presents a set of key technical contributions including the incorporation of receding horizon control, visual conditioning, and the time-series diffusion transformer. We hope this work will help motivate a new generation of policy learning techniques that are able to leverage the powerful generative modeling capabilities of diffusion models. Code, data, and training details are publicly available.
Supplemental materials: https://arxiv.org/abs/2303.04137 , http://diffusion-policy.cs.columbia.edu/
Comments welcome.
Towards Tight Convex Relaxations for Contact-Rich Manipulation
- by Graesdal, Bernhard P and Chia, Shao YC and Marcucci, Tobia and Morozov, Savva and Amice, Alexandre and Parrilo, Pablo A and Tedrake, Russ
We present a method for global motion planning of robotic systems that interact with the environment through contacts. Our method directly handles the hybrid nature of such tasks using tools from convex optimization. We formulate the motion-planning problem as a shortest-path problem in a graph of convex sets, where a path in the graph corresponds to a contact sequence and a convex set models the quasi-static dynamics within a fixed contact mode. For each contact mode, we use semidefinite programming to relax the nonconvex dynamics that results from the simultaneous optimization of the object's pose, contact locations, and contact forces. The result is a tight convex relaxation of the overall planning problem, that can be efficiently solved and quickly rounded to find a feasible contact-rich trajectory. As a first application of this technique, we focus on the task of planar pushing. Exhaustive experiments show that our convex-optimization method generates plans that are consistently within a small percentage of the global optimum. We demonstrate the quality of these plans on a real robotic system.
Supplemental materials: https://arxiv.org/abs/2402.10312
Comments welcome.
Motion planning around obstacles with convex optimization
- by Marcucci, Tobia and Petersen, Mark and von Wrangel, David and Tedrake, Russ
From quadrotors delivering packages in urban areas to robot arms moving in confined warehouses, motion planning around obstacles is a core challenge in modern robotics. Planners based on optimization can design trajectories in high-dimensional spaces while satisfying the robot dynamics. However, in the presence of obstacles, these optimization problems become nonconvex and very hard to solve, even just locally. Thus, when facing cluttered environments, roboticists typically fall back to sampling-based planners that do not scale equally well to high dimensions and struggle with continuous differential constraints. Here, we present a framework that enables convex optimization to efficiently and reliably plan trajectories around obstacles. Specifically, we focus on collision-free motion planning with costs and constraints on the shape, the duration, and the velocity of the trajectory. Using recent techniques for finding shortest paths in Graphs of Convex Sets (GCS), we design a practical convex relaxation of the planning problem. We show that this relaxation is typically very tight, to the point that a cheap postprocessing of its solution is almost always sufficient to identify a collision-free trajectory that is globally optimal (within the parameterized class of curves). Through numerical and hardware experiments, we demonstrate that our planner, which we name GCS, can find better trajectories in less time than widely used sampling-based algorithms and can reliably design trajectories in high-dimensional complex environments.
Comments welcome.
- by Peter Werner and Alexandre Amice and Tobia Marcucci and Daniela Rus and Russ Tedrake
Many computations in robotics can be dramatically accelerated if the robot configuration space is described as a collection of simple sets. For example, recently developed motion planners rely on a convex decomposition of the free space to design collision-free trajectories using fast convex optimization. In this work, we present an efficient method for approximately covering complex configuration spaces with a small number of polytopes. The approach constructs a visibility graph using sampling and generates a clique cover of this graph to find clusters of samples that have mutual line of sight. These clusters are then inflated into large, full-dimensional, polytopes. We evaluate our method on a variety of robotic systems and show that it consistently covers larger portions of free configuration space, with fewer polytopes, and in a fraction of the time compared to previous methods.
Supplemental materials: https://arxiv.org/abs/2310.02875
Under review. Comments welcome.
Locomotion Group News
-
April 22, 2024. PhD Defense. Congratulations to Tobia Marcucci for successfully defending his PhD thesis!
September 19, 2023. Video. Check out our work at TRI on Diffusion Policy (towards Large Behavior Models) and the corresponding blog post
June 14, 2023. Award. Congratulations to Terry Suh, Max Simchowitz, and Kaiqing Zhang who's paper titled "Do differentiable simulators give better policy gradients?" was recognized with the "Best Paper (of the year) Award" from the IEEE RAS Technical Committee on Model-based Optimization for Robotics.
January 20, 2023. PhD Defense. Congratulations to Tao Pang for successfully defending his PhD thesis!
July 15, 2022. Award. Congratulations to Terry Suh, Max Simchowitz, and Kaiqing Zhang who's paper titled "Do differentiable simulators give better policy gradients?" was recognized with the "Outstanding Paper Award" at ICML 2022.
January 13, 2022. Award. Congratulations to Alexandre Amice, Hongkai Dai, Pete Werner, and Annan Zhang who's paper titled "Finding and Optimizing Certified, Collision-Free Regions in Configuration Space for Robot Manipulators" was recognized with the "Outstanding Paper Award" at WAFR 2022.
June 17, 2022. PhD Defense. Congratulations to Yunzhu Li for successfully defending his PhD thesis!
June 17, 2022. PhD Defense. Congratulations to Greg Izatt for successfully defending his PhD thesis!
August 15, 2020. Talks on Zoom. For better or worse, most research talks these days are now online. I've posted a handful of links to new talks, including Russ on Lex Fridman's AI Podcast, and at the IFRR Colloquium on the Roles of Physics-Based Models and Data-Driven Learning in Robotics.
July 20, 2020. PhD Defense. Congratulations to Lucas Manuelli for successfully defending his PhD thesis!
May 29, 2020. PhD Defense. Congratulations to Shen Shen for successfully defending her thesis!
September 18, 2019. PhD Defense. Congratulations to Twan Koolen for successfully defending his thesis!
August 19, 2019. PhD Defense. Congratulations to Pete Florence for successfully defending his thesis!
June 27, 2019. Video. Checkout some of the work being done by TRI's manipulation team
October 15, 2018. PhD Defense. Congratulations to Robin Deits for successfully defending his thesis!
October 3, 2018. Award. Congratulations to Pete Florence and Lucas Manuelli whose paper Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation won the Conference Best Paper Award at CoRL 2018!
September 19, 2018. Award. Congratulations to Pete Florence and Lucas Manuelli whose paper Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation won the first ever Amazon Robotics Best Technical Paper Award (2018).
June 18, 2018. Award. Congratulations to Ani Majumdar whose paper Funnel libraries for real-time robust feedback motion planning won the first ever International Journal of Robotics Research Paper of the Year (2017).
April 26, 2018. Award. Congratulations to Katy Muhlrad for winning the "Audience Choice Award" at the SuperUROP Showcase for her work on "Using GelSight to Identify Objects by Touch".
July 26, 2017. Defense. Frank Permenter successfully defended his thesis, titled "Reduction methods in semidefinite and conic optimization". Congratulations Frank!