The Sound of Movement

If you see the letters on this page in a rainbow of colors, or if hearing a certain word triggers the taste of cigarette smoke, you are a synesthete—one of perhaps every 100 people whose brains are wired in such a way that stimulating one sense activates another as well. Now, Caltech researchers have discovered a type of synesthesia in which sounds, such as tapping, beeping, or whirring, are heard when things move or flash. Synesthesias that trigger the senses of sight, touch, and taste are well known, but an auditory synesthesia had never been identified before.

Caltech postdoc and lecturer in computation and neural systems Melissa Saenz (BS ’98) discovered the phenomenon quite by accident. “While I was running an experiment at the Caltech Brain Imaging Center, a group of students happened to pass by on a tour,” explains Saenz, who, along with Christof Koch, the Troendle Professor of Cognitive and Behavioral Biology and professor of computation and neural systems, reports the finding in the August 5 issue of the journal Current Biology. “My computer screen was showing dots rapidly expanding out, somewhat like the opening scene of Star Wars. Out of the blue, one of the students asked, ‘Does anyone else hear something when you look at that?’ After talking to him further, I realized that his experience had all the characteristics of synesthesia: an automatic sensory cross-activation that he had experienced all of his life.”

Intrigued, Saenz began to look for others with the same ability. (You can view the video—in a quiet location!—here.) “I queried a few hundred people and three more turned up,” she says. “Having that specific example made it easy. It just happens to be quite ‘noisy’ to the synesthetes, and was a great screening tool. When asked if it made a sound, one of the individuals responded, ‘How could it not?’ I would have been less successful had I just generally asked, ‘Do you hear sounds when you see things move or flash?’ because in the real environment, things that move often really do make sounds,” like, for example, buzzing bees.

This may be why the phenomenon hadn’t been detected before—people with it may not realize that their experience is unusual. “These individuals have an enhanced soundtrack in life, rather than a dramatically different sensory experience,” says Saenz.

The four synesthetes outperformed a control group in recognizing patterns of flashes similar to visual Morse code. Such patterns are normally easier to identify as beeps than flashes, so hearing a sound every time you see a flash should give you an advantage. The subjects watched a series of flashes and then had to guess if a second sequence, played afterward, repeated the same pattern. A similar test was given using sequences of beeps. Both the synesthetes and the control group performed equally well when given beeps, but the synesthetes responded correctly to the flashes more than 75 percent of the time, compared to around 50 percent—the level predicted by chance—in the control group.

Saenz and Koch suspect that as much as 1 percent of the population may experience auditory synesthesia. In fact, they think that the brain may normally transfer visual information to the auditory cortex in order to create a prediction of the associated sound. “At this point, very little is known about how the auditory and visual processing systems of the brain work together,” says Saenz, who has begun brain-imaging studies to explore the connection. “Understanding this interaction is important because in normal experience, our senses work together all the time.” —KS