red apple among green red apple among green Visually impaired study subjects were able to pick out the red apple from a plate of green apples. (Photo: Lynn Watson/Shutterstock)

New research could help the blind 'see' colors and shapes

Sensory substitution devices look at the user's surroundings and translate images into sounds.

When one of our five senses ceases to function, can another take its place?

That's the focus of ongoing research out of the Hebrew University of Jerusalem, which aims to allow people with visual impairments to "see" colors or shapes – with their ears.

The research, being conducted by Professor Amir Amedi and his colleagues at the Center for Human Perception and Cognition, focuses on what they call sensory substitution devices (SSDs). These are tools that can help people who are blind or visually impaired to receive information about their environment in a format that they can use.

A paper published recently in the journal Current Biology proved the effectiveness of this technology. The researchers had both blind and blindfolded subjects wear a miniature camera that was connected to a computer or smartphone, which in turn was connected to a set of headphones. The computer translated the images from the camera into what the researchers called "soundscapes," allowing the subjects to pick up and interact with objects in front of them.

In this video of a live demonstration, the blindfolded subject was not only able to pick up a bottle in front of her, she was actually able to choose the green Sprite bottle instead of the orange drink or bottle of Coca-Cola. She could also select the single red apple from a plate full of green apples.

In other demonstrations of this technology, participants could use the SSD "soundscapes" to distinguish people's facial expressions and even read various letters and words.

The research has already yielded a free iPhone app called EyeMusic, which is now marketed as a game to help people "experience the visual world without opening your eyes." The app looks at the user's surroundings and translates different colors into different sounds. The sounds change depending on color and distance, allowing the user to adapt and make choices without looking.

The smartphone app is actually the next big thing for this technology, the researchers say. SSDs are not widely used at this time because existing technologies are too heavy, bulky and expensive. The Hebrew University research shows that SSDs could be brought to a wider audience and in a much more useable and affordable format.

A person employing SSDs will still need to be trained in order to get the full value of these technologies, but the researchers have also shown that the brain is capable of learning to interpret this information. "The human brain is more flexible than we thought," Amedi said. He said the results suggest "that in the blind, brain areas have the potential to be 'awakened' to processing visual properties and tasks even after years or maybe even lifelong blindness, if the proper technologies and training approaches are used."

You can see more demonstrations of these techniques and the related technologies at the Amedi lab's YouTube channel.


Photos and SlideshowsPhotos and Slideshows
New research could help the blind 'see' colors and shapes
Sensory substitution devices, researched in pioneering work out of Hebrew University of Jerusalem, could be affordable aids for the visually impaired.