Correcting Robot Mistakes Using EEG Signals
This project aims to create an intuitive communication channel between humans and robots that uses naturally occurring brain signals - to allow robots to adapt to humans rather than the other way around.
Towards this end, we’ve developed a system that uses EEG signals to correct a robot’s mistakes in real time. For several years researchers have tried to develop robots that can be controlled by brain signals, but many past approaches require the humans to learn to modulate their thoughts in specific patterns that can be mentally tiring. Such systems may involve looking at blinking lights or imagining performing certain motions. This can be especially problematic if we are overseeing a robot to do a dangerous task in manufacturing or construction. Instead, our system simply requires the human to observe the robot and mentally judge whether the robot is making a mistake - a task that is very natural for us to do.
When a person observes a mistake, the brain naturally generates a characteristic signal called an Error-Related Potential (ErrP) signal. Our system decodes EEG data in real time to detect these signals and inform a robot that it should stop or alter its behavior.
In the current experiment, the human observes the Baxter robot perform an object selection task. Baxter reaches towards one of two targets, and the human mentally decides whether or not it made the correct choice. The system monitors the observer's EEG signals, and if it detects an Error-Related Potential it immediately tells Baxter to switch to the other target. In the future, this can be extended to more complex binary-choice tasks and even to more continuous tasks.
Abstract: Communication with a robot using brain activity from a human collaborator could provide a direct and fast feedback loop that is easy and natural for the human, thereby enabling a wide variety of intuitive interaction tasks. This paper explores the application of EEG-measured error-related potentials (ErrPs) to closed-loop robotic control. ErrP signals are particularly useful for robotics tasks because they are naturally occurring within the brain in response to an unexpected error. We decode ErrP signals from a human operator in real time to control a Rethink Robotics Baxter robot during a binary object selection task. We also show that utilizing a secondary interactive error-related potential signal generated during this closed-loop robot task can greatly improve classification performance, suggesting new ways in which robots can acquire human feedback. The design and implementation of the complete system is described, and results are presented for real-time closed-loop and open-loop experiments as well as offline analysis of both primary and secondary ErrP signals. These experiments are performed using general population subjects that have not been trained or screened. This work thereby demonstrates the potential for EEG-based feedback methods to facilitate seamless robotic control, and moves closer towards the goal of real-time intuitive interaction.
Andres F. Salazar-Gomez, Joseph DelPreto, Stephanie Gil, Frank Guenther, Daniela Rus - Correcting Robot Mistakes in Real Time Using EEG Signals
- 2017 IEEE International Conference on Robotics and Automation (ICRA) ,2017
- Pdf BibtexAuthor : Andres F. Salazar-Gomez, Joseph DelPreto, Stephanie Gil, Frank Guenther, Daniela Rus
Title : Correcting Robot Mistakes in Real Time Using EEG Signals
In : 2017 IEEE International Conference on Robotics and Automation (ICRA) -
Date : 2017