Researchers Realize a Vision to Help the Blind

Sometimes the path technology takes to the marketplace is dotted with people who raise a quizzical eyebrow and say, You want to do what? In the case of the Brainport vision device, some of the first people to do so were two of the co-inventors.

The innovation started with Paul Bach-y-Rita, M.D., who was an early pioneer in the field of neuroplasticity — the idea that the brain can be trained to process information in a new way. In the 1960s, he became interested in using that concept to design a device that would transfer the sense of sight into touch, a process known as sensory substitution. He did this by creating a machine out of an old dentist chair with a camera attached. Four hundred little rods popped in and out against a person’s back mimicking the patterns of the objects the camera was seeing.

Though Bach-y-Rita proved that the system worked, not much happened with that invention. When he came to the University of Wisconsin in the 1980s he picked up the idea of sensory substitution again. He was particularly interested in using electrical pulses instead of manual stimulation to represent shapes.

Bach-y-Rita, along with Kurt Kaczmarek, Ph.D., then a staff scientist at the university, designed a system that translated black-and-white images into electrical pulses against a blind person’s fingertip. The strength of the pulse depended on how black, white or gray an object appeared on a computer screen. The device worked well, but it was bulky and cumbersome. Still, it gave people with no sight a way to visually perceive objects for the first time.

“These are truly visual tasks and they did them without their eyes,” says Kaczmarek, now a senior scientist in the university’s Tactile Communication and Neurorehabilitation Lab.

That rudimentary device eventually turned into the Brainport vision device — giving blind people a way to “see” their surroundings through electrical pulses.

Transferring to Tongues

But that device needed to get much smaller, which led to a switch away from fingertip stimulators and the first round of skepticism.

Bach-y-Rita started talking to Kaczmarek and another research colleague, Mitchell Tyler, about using people’s tongues instead of fingertips. His reasoning was that the tongue is super sensitive, a large part of the brain is devoted to processing information from the tongue, and except for when we’re eating and talking, the tongue doesn’t do a whole lot.

“We looked at him kind of funny,” Kaczmarek says. “Mitch and I thought he was being a little crazy.”

Bach-y-Rita, it turned out, was onto something. He spent a couple years mulling over ways to use the tongue for sensory substitution. Kaczmarek and Tyler finally acquiesced and agreed to give it a try. They took the fingertip device, and stuck it on their tongues. The pulses it delivered were comfortable and effective. People have described it as feeling like champagne bubbles are painting a picture in their mouth. The researchers did some preliminary studies and proved that people could recognize basic geometric shapes while wearing it, and it performed as well as the fingertip version.

One of the key “aha moments” Kaczmarek recalls was realizing that the tongue required much less circuitry than fingertip devices. The surface of our skin changes depending on whether we’re hot, cold or sweating. Tongues stay uniformly wet and warm. So the electrical circuitry needed to generate and control the pulses is much simpler, meaning the device can be much smaller. Kaczmarek, who designed the first tactile tongue display, got the hardware down to the size of a shoebox. It would eventually become the size of a cell phone.

Road to Commercialization

The group published its results in 1998 and approached the school’s private, nonprofit technology transfer organization, the Wisconsin Alumni Research Foundation (WARF), about patenting the invention. WARF eagerly proceeded. Then it started looking for companies interested in licensing the technology. It found none.

“It was a little off the wall — thinking about seeing with your tongue,” says Jeanine Burmania, a licensing manager with WARF. She says the market didn’t fully appreciate the device’s potential at the time.

Or, as Kaczmarek says: “We were viewed as those crazy people in Madison who were doing things with the tongue.”

So Bach-y-Rita, who passed away in 2006, decided to found a company called Wicab, after his wife’s maiden name.

“It’s a good example of the passion of an inventor who wanted to see his technology commercialized,” Burmania says.

WARF exclusively licensed the technology to Wicab in 1999. WARF received equity from Wicab in lieu of an upfront licensing fee, an arrangement WARF frequently makes with startups, Burmania says.

“Sometimes, technologies developed at universities need further development prior to attracting interest from existing companies,” she continues. “This development is often outside the scope of traditional university funding mechanisms and no longer feasible within the university setting. The Brainport technology is an example of a device that could have languished in the lab from lack of exposure, but was able to make it out of the university because of Wicab.”

For the first several years, Wicab operated mostly as a research and development company, and the personnel overlapped entirely with the university staff.

A couple years into their work with Wicab, the team discovered a new way to use the tactile tongue display. Tyler had a bad inner-ear infection, which caused him to have balance problems. The group decided to try putting a sensor on a helmet that monitored tilt and then translated that information to people’s tongue via electrical pulses that let them know if they were off balance. After seeing that it worked, they started developing a balance device in addition to the vision device.

In 2005, Robert Beckman took over as chief executive officer of Wicab. A veteran of other medical device companies,

Bach-y-Rita invited him to helm the still fledgling company. Beckman realized that the commercial viability of the Brainport balance device was much higher than the vision device because there are many more people with balance problems than those who are totally blind. So they pushed ahead aggressively with the balance device, using much of the $10 million he raised from angel investors during his first year on the job. The vision device was still being developed, mostly with the help of funding from the National Institutes of Health, the National Eye Institute, the Defense Advanced Research Projects Agency and the State of Pennsylvania.

Then Wicab got some bad news. The company had received approval in Europe and Canada to sell the device, but that process is based only on proving the safety of the device not its efficacy. During U. S. Food and Drug Administration (FDA) trials earlier this year, the company discovered that its balance device, while effective 60 percent of the time, was not more effective than the “sham device” that half of the people in the study used. Beckman believes the benefit came from the training and exercises that both groups of users went through in conjunction with their participation in the clinical study. Kaczmarek isn’t convinced that it was the training and exercises alone that helped 60 percent of the people in the study and is trying to puzzle this out in his lab.

Either way, Beckman says of the results: “It’s good for science. It’s bad for commercialization.”

Independence Through Improved Sight

Wicab’s full attention has now turned to the vision device, which could get FDA approval as early as the end of the year, Beckman says. They have proved that the device works by testing it on more than 100 people who have had great success with it.

While there are 300,000 people with no sight in the United States, the company is focusing only on the 100,000 who are not elderly and, therefore, probably more receptive to new technology. He’s also interested in the potential market in China and India.

The device has shrunk considerably since its days in the University of Wisconsin lab. Users now wear a pair of sunglasses with a camera mounted on the nose bridge. A lollipop-sized square, which has 400 electrodes in it, sits on their tongue. The stimulation pattern of those electrodes mimic whatever the camera is picking up, essentially acting as the camera’s pixels. So if the camera is seeing white, the user gets a stronger pulse, gray gets a medium pulse and black gets no pulse. A cell phone-sized control box is attached by a wire to the camera and allows the user to zoom in and out on specific objects. Beckman says the next step is to make the device wireless.

Users get trained for 10 hours on the device, and almost all are very comfortable with it at the end and eager to take it outside or use it at home, says Aimee Arnoldussen, Ph.D., a neuroscientist with Wicab. They’ve listed the many tasks and activities they would love to use the device for, from simply reaching directly for a cup of coffee on the table instead of having to feel around for it to being able to run a marathon by following a guide instead of having to be tethered to him.

“They talk about the independence that the devices can give them,” Arnoldussen says. Unlike many devices for the blind that read aloud the information people can’t see, the Brainport allows people to dictate what they want to pay attention to.

“The user gets to control this technology and what information they’d like to understand,” she says. “They get to decide where their attention is drawn.”

In a video on Wicab’s Web site, blind adventurer Erik Weihenmayer demonstrates how he can rock climb while wearing the device. Weihenmayer, who lost his vision as a teenager, has climbed Mount Everest and scaled El Capitan since losing his sight. He also performs some less daring feats in the video, such as playing tic-tac-toe with his daughter while wearing the Brainport.

Some of the most emotional users, Arnoldussen says, are military personnel who lost their sight in explosions in Iraq and Afghanistan. “In many ways they’ve given up having visual perception, and we can provide that for them,” she says.

The Rotary Foundation has bought early versions of the vision device to help blind children in Central and South America. “Can you imagine the experience of a child who hasn’t had sight before being able to comprehend objects around them?” Arnoldussen says.

Who would raise a quizzical eyebrow to that?


This story was originally published in 2010.

To see available technologies from research institutions, click here to visit the AUTM Innovation Marketplace.