vOICe

The vOICe is a sensory substitution device that translates images to sounds for the blind to allow them to understand a visual experience using other senses. Sights, sounds, and smells integrate and interact within the brain so a sight can be “seen” in ways other than just the eyes. This works through multi-sensory interaction or multi-modal processing, activating the visual cortex through sound. With this device, cross-modal mappings stimulates the users visual cortex with auditory signals that encode information about the environment.

The device is made up of a small computer connected to a camera that is attached to darkened glasses, allowing it to “see” what a human eye would. A computer algorithm scans each camera image from left to right, and for every column of pixels, generates an associated sound with a frequency. A large number of bright pixels at the top of a column translates that image into a loud, high-frequency sound, whereas a large number of lower dark pixels would be a quieter, lower-pitched sound. A user who is blind that wears the camera on a pair of glasses could then associate different sounds with features of their environment. The intuitively identified textures used in the experiments exploited the cross-modal mappings.