Science in the Triangle

Photo courtesy of Tim Bunce.

Nothing raises your heart rate quite like an encroaching horde of zombie skeleton warriors. Throw in a squadron of well-fortified goblin archers and a few steadily advancing mutant spider-crabs, and you’ve got all the ingredients for a high-stress situation — even if it is in a virtual world.

But Alan Pope and Chad Stephens want to help gamers become a little more zen about this kind of thing.

The two NASA researchers, who specialize in aerospace technology and human-machine interfaces, have developed a biofeedback add-on for the Wii controller that can measure key stress indicators during gameplay. Dubbed “MindShift,” the device can alter performance based on those stress levels, effectively allowing gamers to calm their minds through training.

“It amplifies or magnifies a person’s emotions, a person’s boredom and makes it disruptive in the game,” Pope said. “You have to overcome that disruption by achieving a more positive emotional state or better focus and concentration.”

The device consists mostly of off-the-shelf equipment like monitors for the ear, finger and chest. It can even accommodate brain wave caps, according to Brent Fagg, innovation manager at the Triangle-based RTI International. His firm is marketing licenses for the patent-pending device to peripheral manufacturers.

“NASA’s goal is to get it into the public’s hands rather than make a profit,” he said.

Before it gets to consumers though, Pope said the hardware will need a little refining. But he says the technology has come a long way over the last two decades.

RESEARCH IN MOTION

When he started exploring the concept, Pope used biofeedback systems to measure pilot stress in training. Those measurements then fed back into the flight simulation, increasing or decreasing the difficulty based on changes in the test subject’s stress level. Pope applied the same concept to the PlayStation to help children with ADHD, then licensed the patent to SmartBrain Technologies.

“It occurred to me in the mid-90s that the settings we were working with — flight simulators — were a lot like video games. So I started thinking about how I could use this idea of connecting physiological signals to systems in the context of video games,” Pope said. “It was a pretty easy translation.”

While he now considers that early demonstration “sort of rudimentary,” the Nintendo Wii console presented a new challenge. Of particular interest to the team was the new controller scheme.

“The Wii, the way it’s designed lends itself to modification,” Stephens said. “It also taps into a different modality of motion — instead of pressing buttons to get a character to do anything, it uses motion controllers.”

The inspiration for using the Wii actually came from the team’s then-high school intern, Nina Blanson, who’s now a student at Yale.

“When we put several ideas in front of her to ask which one of these is of interest to [her], it didn’t take her long to select the Wii video game as the one she would be interested in pursuing,” Pope said.

After months of brainstorming, Pope said they found a solution that affects gameplay “from the outside,” meaning they needed to install no software patches or obtain any proprietary information from Nintendo.

“We had to play with it for a while, go down a couple of blind alleys, before really coming up with a neat way of doing it,” Pope said.

The researchers declined to detail exactly how MindShift works, pointing out they’re still in the middle of patenting the technology. But they said it is already compatible with every Wii game off the shelf.

“It’s not always going to be as good with some games as with others,” Fagg said. “It’s the kind of thing where five of them will be awesome, five of them are going to suck.”

They said results have been impressive with the games they’ve tested so far. That includes Link’s Crossbow Training, a first-person shooter; Trauma Center, a surgery simulator; and Wii Sports Golf, which is packaged with the Wii console.

“In that game, we can actually connect a player’s state to their swing strength, and if they’re not in an optimal state, then their swing will be greatly reduced,” Stephens said.

The team also envisions pairing its technology with downloadable content and other add-ons, effectively increasing the value of the original game title.

“The player coming to the game that they’re best in the world at would face an additional challenge, another layer of challenge, in performing the game and playing the best they could,” Stephens said.

‘A DIFFERENT SKILLSET’

Aside from adding a whole new element of gameplay, Stephens said MindShift injects more realism into the gaming experience. Instead of measuring a driver’s nervousness or a sniper’s unsteadiness by a generic algorithm in the gaming code for example, this device could incorporate the actual mental state of the user.

“When you’re interfacing with any machine — whether it’s a video game, vehicle or computer — you sort of have a disconnect between your internal state and your external behavior,” Stephens said. “What this technology is trying to do is add that layer back into that interaction.”

It can also serve as a draw for nongamers not used to conventional controllers. Most highly competitive games require expert hand motion almost second nature to lifetime gamers.

“You might be very good with your hands and your thumbs — which is what current controllers require, a lot of thumb-twitch action,” Pope said. “But when you pick up the MindShift video game technology, you’d have to develop a different skillset: a skillset of controlling your internal psychological or mental state.”

That element could potentially level the playing field.

“It doesn’t depend on how cracked out you are on Mountain Dew,” Fagg said with a laugh.

But the effects of the biofeedback won’t last forever. The ultimate goal, Pope said, is to teach users to control their mental state and calm their nerves, even in high-stress situations.

“Eventually, when you learn that control, then there’s no difference in playing the game with your hand like you would without using this technology,” Pope said.

While RTI works to connect the technology to peripheral manufacturers like Mad Catz Inc., Logitech or Nyko, Pope and Stephens will work to expand to the PlayStation Move and Xbox Kinect. They’re even scheduled to speak about their work at TEDxNASA Nov. 4.

“Once you get into the mindset of thinking of this kind of thing, then there’s a whole domain of ways to do it that start occurring to you,” Pope said. “We’re hoping those will continue to unfold.”

Tyler Dukes is a freelance reporter and journalism adviser at N.C. State University. Follow him on Twitter as @mtdukes.