Home
People
Publications
Talks/Lectures
Video Highlights
Robotics Challenge
Software
Directions
Accessibility
Robot Locomotion Group
The goal of our research is to build machines which exploit their natural dynamics to achieve extraordinary agility, efficiency, and robustness using rigorous tools from dynamical systems, control theory, and machine learning. Our current focus in on robotic manipulation, because the revolution in recent machine learning has opened a pathway in these applications to merging control theory and perception at a level that has never been considered before; ideas like "intuitive physics" and "common-sense reasoning" will meet with rigorous ideas like "model-order reduction" and "robust/adaptive control". It's going to be a great few years!
Our previous projects have included dynamics and control for humanoid robots, dynamic walking over rough terrain, flight control for aggressive maneuvers in unmanned aerial vehicles, feedback control for fluid dynamics and soft robotics, and connections between perception and control.
The Robot Locomotion Group is a part of Robotics @ MIT and CSAIL.
Locomotion Group Paper and Multimedia News 
-
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
- by Cheng Chi and Zhenjia Xu and Siyuan Feng and Eric Cousineau and Yilun Du and Benjamin Burchfiel and Russ Tedrake and Shuran Song
This paper introduces Diffusion Policy, a new way of generating robot behavior by representing a robot's visuomotor policy as a conditional denoising diffusion process. We benchmark Diffusion Policy across 12 different tasks from 4 different robot manipulation benchmarks and find that it consistently outperforms existing state-of-the-art robot learning methods with an average improvement of 46.9%. Diffusion Policy learns the gradient of the action-distribution score function and iteratively optimizes with respect to this gradient field during inference via a series of stochastic Langevin dynamics steps. We find that the diffusion formulation yields powerful advantages when used for robot policies, including gracefully handling multimodal action distributions, being suitable for high-dimensional action spaces, and exhibiting impressive training stability. To fully unlock the potential of diffusion models for visuomotor policy learning on physical robots, this paper presents a set of key technical contributions including the incorporation of receding horizon control, visual conditioning, and the time-series diffusion transformer. We hope this work will help motivate a new generation of policy learning techniques that are able to leverage the powerful generative modeling capabilities of diffusion models. Code, data, and training details are publicly available.
Supplemental materials: https://arxiv.org/abs/2303.04137 , http://diffusion-policy.cs.columbia.edu/
Comments welcome.
Towards Tight Convex Relaxations for Contact-Rich Manipulation
- by Graesdal, Bernhard P and Chia, Shao YC and Marcucci, Tobia and Morozov, Savva and Amice, Alexandre and Parrilo, Pablo A and Tedrake, Russ
We present a method for global motion planning of robotic systems that interact with the environment through contacts. Our method directly handles the hybrid nature of such tasks using tools from convex optimization. We formulate the motion-planning problem as a shortest-path problem in a graph of convex sets, where a path in the graph corresponds to a contact sequence and a convex set models the quasi-static dynamics within a fixed contact mode. For each contact mode, we use semidefinite programming to relax the nonconvex dynamics that results from the simultaneous optimization of the object's pose, contact locations, and contact forces. The result is a tight convex relaxation of the overall planning problem, that can be efficiently solved and quickly rounded to find a feasible contact-rich trajectory. As a first application of this technique, we focus on the task of planar pushing. Exhaustive experiments show that our convex-optimization method generates plans that are consistently within a small percentage of the global optimum. We demonstrate the quality of these plans on a real robotic system.
Supplemental materials: https://arxiv.org/abs/2402.10312
Comments welcome.
Motion planning around obstacles with convex optimization
- by Marcucci, Tobia and Petersen, Mark and von Wrangel, David and Tedrake, Russ
From quadrotors delivering packages in urban areas to robot arms moving in confined warehouses, motion planning around obstacles is a core challenge in modern robotics. Planners based on optimization can design trajectories in high-dimensional spaces while satisfying the robot dynamics. However, in the presence of obstacles, these optimization problems become nonconvex and very hard to solve, even just locally. Thus, when facing cluttered environments, roboticists typically fall back to sampling-based planners that do not scale equally well to high dimensions and struggle with continuous differential constraints. Here, we present a framework that enables convex optimization to efficiently and reliably plan trajectories around obstacles. Specifically, we focus on collision-free motion planning with costs and constraints on the shape, the duration, and the velocity of the trajectory. Using recent techniques for finding shortest paths in Graphs of Convex Sets (GCS), we design a practical convex relaxation of the planning problem. We show that this relaxation is typically very tight, to the point that a cheap postprocessing of its solution is almost always sufficient to identify a collision-free trajectory that is globally optimal (within the parameterized class of curves). Through numerical and hardware experiments, we demonstrate that our planner, which we name GCS, can find better trajectories in less time than widely used sampling-based algorithms and can reliably design trajectories in high-dimensional complex environments.
Comments welcome.
- by Peter Werner and Alexandre Amice and Tobia Marcucci and Daniela Rus and Russ Tedrake
Many computations in robotics can be dramatically accelerated if the robot configuration space is described as a collection of simple sets. For example, recently developed motion planners rely on a convex decomposition of the free space to design collision-free trajectories using fast convex optimization. In this work, we present an efficient method for approximately covering complex configuration spaces with a small number of polytopes. The approach constructs a visibility graph using sampling and generates a clique cover of this graph to find clusters of samples that have mutual line of sight. These clusters are then inflated into large, full-dimensional, polytopes. We evaluate our method on a variety of robotic systems and show that it consistently covers larger portions of free configuration space, with fewer polytopes, and in a fraction of the time compared to previous methods.
Supplemental materials: https://arxiv.org/abs/2310.02875
Under review. Comments welcome.
Constrained Bimanual Planning with Analytic Inverse Kinematics
- by Thomas Cohn and Seiji Shaw and Max Simchowitz and Russ Tedrake
In order for a bimanual robot to manipulate an object that is held by both hands, it must construct motion plans such that the transformation between its end effectors remains fixed. This amounts to complicated nonlinear equality constraints in the configuration space, which are difficult for trajectory optimizers. In addition, the set of feasible configurations becomes a measure zero set, which presents a challenge to sampling-based motion planners. We leverage an analytic solution to the inverse kinematics problem to parametrize the configuration space, resulting in a lower-dimensional representation where the set of valid configurations has positive measure. We describe how to use this parametrization with existing algorithms for motion planning, including sampling-based approaches, trajectory optimizers, and techniques that plan through convex inner-approximations of collision-free space.
Under review. Comments welcome.
Locomotion Group News
-
September 19, 2023. Video. Check out our work at TRI on Diffusion Policy (towards Large
Behavior Models) and the corresponding blog post
June 14, 2023. Award. Congratulations to Terry Suh, Max Simchowitz, and Kaiqing Zhang who's paper titled "Do differentiable simulators give better policy gradients?" was recognized with the "Best Paper (of the year) Award" from the IEEE RAS Technical Committee on Model-based Optimization for Robotics.
January 20, 2023. PhD Defense. Congratulations to Tao Pang for successfully defending his PhD thesis!
July 15, 2022. Award. Congratulations to Terry Suh, Max Simchowitz, and Kaiqing Zhang who's paper titled "Do differentiable simulators give better policy gradients?" was recognized with the "Outstanding Paper Award" at ICML 2022.
January 13, 2022. Award. Congratulations to Alexandre Amice, Hongkai Dai, Pete Werner, and Annan Zhang who's paper titled "Finding and Optimizing Certified, Collision-Free Regions in Configuration Space for Robot Manipulators" was recognized with the "Outstanding Paper Award" at WAFR 2022.
June 17, 2022. PhD Defense. Congratulations to Yunzhu Li for successfully defending his PhD thesis!
June 17, 2022. PhD Defense. Congratulations to Greg Izatt for successfully defending his PhD thesis!
August 15, 2020. Talks on Zoom. For better or worse, most research talks these days are now online. I've posted a handful of links to new talks, including Russ on Lex Fridman's AI Podcast, and at the IFRR Colloquium on the Roles of Physics-Based Models and Data-Driven Learning in Robotics.
July 20, 2020. PhD Defense. Congratulations to Lucas Manuelli for successfully defending his PhD thesis!
May 29, 2020. PhD Defense. Congratulations to Shen Shen for successfully defending her thesis!
September 18, 2019. PhD Defense. Congratulations to Twan Koolen for successfully defending his thesis!
August 19, 2019. PhD Defense. Congratulations to Pete Florence for successfully defending his thesis!
June 27, 2019. Video. Checkout some of the work being done by TRI's manipulation team
October 15, 2018. PhD Defense. Congratulations to Robin Deits for successfully defending his thesis!
October 3, 2018. Award. Congratulations to Pete Florence and Lucas Manuelli whose paper Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation won the Conference Best Paper Award at CoRL 2018!
September 19, 2018. Award. Congratulations to Pete Florence and Lucas Manuelli whose paper Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation won the first ever Amazon Robotics Best Technical Paper Award (2018).
June 18, 2018. Award. Congratulations to Ani Majumdar whose paper Funnel libraries for real-time robust feedback motion planning won the first ever International Journal of Robotics Research Paper of the Year (2017).
April 26, 2018. Award. Congratulations to Katy Muhlrad for winning the "Audience Choice Award" at the SuperUROP Showcase for her work on "Using GelSight to Identify Objects by Touch".
July 26, 2017. Defense. Frank Permenter successfully defended his thesis, titled "Reduction methods in semidefinite and conic optimization". Congratulations Frank!
May 19, 2017. Award. Pete Florence was awarded the EECS Masterworks award. Congratulations Pete!