A team of Rensselaer Polytechnic Institute students has created a system that pairs an EEG headset with a 3-D theatrical flying harness, allowing users to “fly” by controlling their thoughts.
The “Infinity Simulator” will make its debut with an art installation in which participants rise into the air – and trigger light, sound, and video effects – by calming their thoughts.
Creative director and Rensselaer MFA candidate Yehuda Duenyas describes the “Infinity Simulator” as a platform similar to a gaming console — like the Wii or the Kinect — writ large.
“Instead of you sitting and controlling gaming content, it’s a whole system that can control live elements — so you can control 3-D rigging, sound, lights, and video,” said Duenyas, who works under the moniker “xxxy.”
“It’s a system for creating hybrids of theater, installation, game, and ride.”
Duenyas created the “Infinity Simulator” with a team of collaborators, including Michael Todd, a Rensselaer 2010 graduate in computer science. Duenyas will exhibit the new system in the art installation “The Ascent” on May 12 at Curtis R. Priem Experimental Media and Performing Arts Center (EMPAC).
Ascent works with an Emotiv EEG headset. Using the headset the user can imagine how they want to move/fly and the data is then interpreted and fed into the 3D automated rigging system which then raises, lowers and steers the user accordingly.
Ten computer programs running simultaneously link the commercially available EEG headset to the computer-controlled 3-D flying harness and various theater systems, said Todd.
This is the first test of the Infinity System, powered by custom software, Arduino, electronic compass and accelerometer. We changed some scripts in the Unity® Island Demo so you can fly through the island.
“We’ve built a software system on top of the rigging control board and now have control of it through an iPad, and since we have the iPad control, we can have anything control it,” said Duenyas. “The ‘Infinity Simulator’ is the center; everything talks to the ‘Infinity Simulator.’”
The May 12 “The Ascent” installation is only one experience made possible by the new platform, Duenyas said.
“‘The Ascent’ embodies the maiden experience that we’ll be presenting,” Duenyas said. “But we’ve found that it’s a versatile platform to create almost any type of experience that involves rigging, video, sound, and light. The idea is that it’s reactive to the users’ body; there’s a physical interaction.”
“The thing that I focus on the most is user experience,” Duenyas said. “All the shows I do with my theater company and on my own involve a lot of set and set design – you’re entering into a whole world. You’re having an experience that is more than going to a show, although a show is part of it.”
At Rensselaer, Duenyas took advantage of the technology at hand to explore his idea, first with a video game he developed in 2010, then — working through the Department of the Arts — with EMPAC’s computer-controlled 3-D theatrical flying harness. The 2010 video game, also developed with Todd, tracked the movements — pitch and yaw — of players suspended in a custom-rigged harness, allowing players to soar through simulated landscapes. Duenyas said that that game (also called the “Infinity Simulator”) and the new platform are part of the same vision.
EMPAC Director Johannes Goebel saw the game on display at the 2010 GameFest and discussed the custom-designed 3-D theatrical flying rig in EMPAC with Duenyas. Working through the Arts Department, Duenyas submitted a proposal to work with the rig, and his proposal was accepted.
Duenyas and his team experimented — first gaining peripheral control over the system, and then linking it to the EEG headset – and created the Ascent installation as an initial project. In the installation, the Infinity Simulator is programmed to respond to relaxation.
“We’re measuring two brain states – alpha and theta — waking consciousness and everyday brain computational processing,” said Duenyas. “If you close your eyes and take a deep breath, that processing power decreases. When it decreases below a certain threshold, that is the trigger for you to elevate.”
As a user rises, their ascent triggers a changing display of lights, sound, and video. Duenyas said he wants to hint at transcendental experience, while keeping the door open for a more circumspect interpretation.
“The point is that the user is trying to transcend the everyday and get into this meditative state so they can have this experience. I see it as some sort of iconic spiritual simulator. That’s the serious side,” he said. “There’s also a real tongue-in-cheek side of my work: I want clouds, I want Terry Gilliam’s animated fist to pop out of a cloud and hit you in the face. It’s mixing serious religious symbology, but not taking it seriously.”
“It started with, ‘I want to have a glimpse of infinity,’ ‘I want to float in space.’ Then you get in the harness and you’re like ‘man, this harness is uncomfortable,’” he said. “In order to achieve the original vision, we had to build an infrastructure, and I still see development of the infinity experience is a ways off; but what we can do with the infrastructure in a realistic time frame is create ‘The Ascent,’ which is going to be really fun, and totally other.”
Creating the “Infinity Simulator” has prompted new possibilities.
“The vision now is to play with this fun system that we can use to build any experience,” he said. “It’s sort of overwhelming because you could do so many things – you could create a flight through cumulus clouds, you could create an augmented physicality parkour course where you set up different features in the room and guide yourself to different heights. It’s limitless.”