MIT finger device reads to the blind in real time

ShutterStock image

CAMBRIDGE, Mass. — Scientists at the Massachusetts Institute of Technology are developing an audio reading device to be worn on the index finger of people whose vision is impaired, giving them affordable and immediate access to printed words.

The so-called FingerReader, a prototype produced by a 3-D printer, fits like a ring on the user’s finger, equipped with a small camera that scans text. A synthesized voice reads words aloud, quickly translating books, restaurant menus and other needed materials for daily living, especially away from home or office.

Reading is as easy as pointing the finger at text. Special software tracks the finger movement, identifies words and processes the information. The device has vibration motors that alert readers when they stray from the script, said Roy Shilkrot, who is developing the device at the MIT Media Lab.

For Jerry Berrier, 62, who was born blind, the promise of the FingerReader is its portability and offer of real-time functionality at school, a doctor’s office and restaurants.

“When I go to the doctor’s office, there may be forms that I wanna read before I sign them,” Berrier said.

He said there are other optical character recognition devices on the market for those with vision impairments, but none that he knows of that will read in real time.

Berrier manages training and evaluation for a federal program that distributes technology to low-income people in Massachusetts and Rhode Island who have lost their sight and hearing. He works from the Perkins School for the Blind in Watertown, Massachusetts.

“Everywhere we go, for folks who are sighted, there are things that inform us about the products that we are about to interact with. I wanna be able to interact with those same products, regardless of how I have to do it,” Berrier said.

Pattie Maes, an MIT professor who founded and leads the Fluid Interfaces research group developing the prototype, says the FingerReader is like “reading with the tip of your finger and it’s a lot more flexible, a lot more immediate than any solution that they have right now.”

Developing the gizmo has taken three years of software coding, experimenting with various designs and working on feedback from a test group of visually impaired people. Much work remains before it is ready for the market, Shilkrot said, including making it work on cellphones.

Shilkrot said developers believe they will be able to affordably market the FingerReader but he could not yet estimate a price. The potential market includes some of the 11.2 million people in the United States with vision impairment, according to U.S. Census Bureau estimates.

Current technology used in homes and offices offers cumbersome scanners that must process the desired script before it can be read aloud by character-recognition software installed on a computer or smartphone, Shilkrot said. The FingerReader would not replace Braille — the system of raised dots that form words, interpreted by touch. Instead, Shilkrot said, the new device would enable users to access a vast number of books and other materials that are not currently available in Braille.

Developers had to overcome unusual challenges to help people with visual impairments move their reading fingers along a straight line of printed text that they could not see. Users also had to be alerted at the beginning and end of the reading material.

Their solutions? Audio cues in the software that processes information from the FingerReader and vibration motors in the ring.

The FingerReader can read papers, books, magazines, newspapers, computer screens and other devices, but it has problems with text on a touch screen, said Shilkrot.

That’s because touching the screen with the tip of the finger would move text around, producing unintended results. Disabling the touch-screen function eliminates the problem, he said.

Berrier said affordable pricing could make the FingerReader a key tool to help people with vision impairment integrate into the modern information economy.

“Any tool that we can get that gives us better access to printed material helps us to live fuller, richer, more productive lives, Berrier said.