Monkeys in Nicolelis lab control both arms on a virtual avatar using just their brain activity
The Nicolelis lab is turning science fiction into reality again by making it possible for two monkeys to control two virtual arms of a monkey avatar on a computer screen without moving any part of their real bodies. The study, released from the lab last week, is part of the broader Walk Again Project aiming to build exoskeletal suits that can be controlled by paralyzed patients using the brain-machine interface technology.
According to Duke University’s press release it is the first-ever two-way bimanual interaction between a primate brain and a virtual body. Two years ago in 2011, the same team demonstrated a unique two-way interface between mind and machine, but back then the work only let monkeys control a single limb. Now the latest research allows monkeys to control two virtual arms with their thoughts alone.
“Bimanual movements in our daily activities — from typing on a keyboard to opening a can — are critically important,” said Nicolelis in a statement. “Future brain-machine interfaces aimed at restoring mobility in humans will have to incorporate multiple limbs to greatly benefit severely paralyzed patients.”
The monkeys were trained in a virtual environment within which they viewed realistic avatar arms on a screen and were encouraged to place their virtual hands on specific targets in a bimanual motor task. The monkeys first learned to control the avatar arms using a pair of joysticks, but were able to learn to use just their brain activity to move both avatar arms without moving their own arms.
Virtual monkey avatar shown from a 3rd person perspective as the movements of the two arms are decoded in real-time from the brain of a rhesus monkey. In the experiment the virtual arms and 3D target objects appear on the screen from a first-person perspective to the monkey, who receives a juice reward for correctly performed trials. Source: Nicolelis lab
Screen as viewed by the monkey during experiments: a first-person perspective of the rhesus monkey avatar limbs. The movements of both virtual arms are decoded in real-time from brain activity while the monkey’s own arms were not permitted to freely move. The monkey must move the virtual arms to the circular targets to receive a small juice reward. Source: Nicolelis lab
The new research isn’t just spectacular but also gives further support to the theory that large neuronal ensembles, not single neurons, control motor functions. In order to record a larger number of signals required for controlling bimanual movements in monkeys, the existing technology had to be innovated, says Peter Ifft, a biomedical engineering graduate student in the Nicolelis lab and lead author of this recent study.
“We wrote an algorithm that can handle large numbers of channels of up to 500 cells,” Ifft said. “It can decode the signal in real time with 100 millisecond resolution.”
Ifft recorded nearly 500 cells simultaneously, which is the highest number recorded to date.
The Nicolelis lab
Miguel Nicolelis, co-director of the Center of Neuroengineering at Duke University, founded his laboratory in 1994 to understand how large populations of brain cells interact in behaving animals.
Two years later they began working with monkeys and developed their technology to record much larger populations of neurons. Back then, it was believed that single neurons were the key functional units of the brain, but using the brain-imaging interface technology, the lab was able to demonstrate that a large network of neurons, not a single neuronal pathway, compose functional units in the brain.
The lab currently consists of approximately 50 people including staff, undergraduates, graduate students and PhD candidates. In addition to pioneering the field of neuroprosthetics, Nicolelis is also a strong advocate of applying science education and technology in the service of society.
He established the Edmond and Lily Safra International Institute for Neuroscience of Natal in Brazil which serves as a research center, but also provides services to the local community. Along with research, the Institute features a middle school and high school where students can learn science through a hands on approach. There is also a women’s healthcare clinic that does research on high risk pregnancies and provides free prenatal care to over 12,000 women each year.
Nicolelis is also involved in the Walk Again Project, an effort to develop brain-controlled mobility devices for paralyzed people. The project is working on a whole-body exoskeleton that would be controlled by the wearer’s thoughts. The technology is set to debut in a demonstration at the 2014 FIFA World Cup in Brazil.