Jump to content
IGNORED

Interaction Between Vision and Hearing Isn't Just One Way


Recommended Posts

I've seen posts from time to time about the McGurk effect, and thought this article about information traveling in the other direction might interest people. From the journal Current Biology, here's the abstract:

 

Hearing sounds helps our visual system to predict incoming information and may give us a survival advantage. Vetter et al. blindfolded people and scanned their brains while they listened to birds singing, traffic noise, or people talking. Using sophisticated algorithms, the researchers were able to identify the category of sounds just by examining the pattern of activity in the primary visual cortex, a brain area previously believed to process nothing but input from the eyes. And when the people imagined the specific sound categories in the complete absence of sight and sound, their primary visual cortices also showed activity. These results highlight the interconnectedness of the brain's sensory systems.

One never knows, do one? - Fats Waller

The fairest thing we can experience is the mysterious. It is the fundamental emotion which stands at the cradle of true art and true science. - Einstein

Computer, Audirvana -> optical Ethernet to Fitlet3 -> Fibbr Alpha Optical USB -> iFi NEO iDSD DAC -> Apollon Audio 1ET400A Mini (Purifi based) -> Vandersteen 3A Signature.

Link to comment
I've seen posts from time to time about the McGurk effect, and thought this article about information traveling in the other direction might interest people.

Definitely! The evidence strongly suggests that non-visual sensory stimulus evokes a mental "picture" that's processed in the visual cortex even without a visual stimulus. It appears that we "see" what we imagine in much the same way that we see what we see. There's been a lot of work strongly suggesting that we process remembered or imagined visual images in the visual cortex, e.g. this 1993 NIH study. So it's not at all a reach to say that we also "see" images stimulated by smells, sounds and tactile input.

 

There's a very new study from the Max Planck Institute showing clearly that " [E]ven in the absence of visual stimuli, the visual cortex is differentially activated depending on the position of an acoustic sound source. Our results show that the visual cortex receives information about the position of auditory stimuli within the visual field......[C]ortical sensory areas receive information about the spatial position of objects of a different modality. Our study shows that cross-modally induced reorganization of the visual oscillatory activity depends on the position of a sound source. "

 

This is probably why we "see" a sound stage between and around our speakers and headphones. The orchestra "appears" as we know or imagine it to be - the first violins front left, cellos front right, trumpets center rear etc. And we have a problem when what we hear is not completely consistent with our expectations - I think of it as audiovisual cognitive dissonance.

 

When we "see" Ella or Sarah or Frank standing between our speakers with our eyes closed, it's almost certainly the visual cortex that's firing on all fours to make this happen.

Link to comment
And we have a problem when what we hear is not completely consistent with our expectations - I think of it as audiovisual cognitive dissonance.

 

 

--which is quite possibly a reason for "listening fatigue" - the brain is perhaps working overtime trying to make things fit when they don't.

One never knows, do one? - Fats Waller

The fairest thing we can experience is the mysterious. It is the fundamental emotion which stands at the cradle of true art and true science. - Einstein

Computer, Audirvana -> optical Ethernet to Fitlet3 -> Fibbr Alpha Optical USB -> iFi NEO iDSD DAC -> Apollon Audio 1ET400A Mini (Purifi based) -> Vandersteen 3A Signature.

Link to comment
--which is quite possibly a reason for "listening fatigue" - the brain is perhaps working overtime trying to make things fit when they don't.

Wow - that's insight, Jud! Maybe our brains really can hurt when in distress. It would certainly explain why the usual metrics like frequency response don't correlate. And even though I've thought about this for years, I never considered that it might actually have a physical analogue.

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...