The ventriloquism illusion does modulate the Mismatch Negativity (MMN) (original) (raw)
Related papers
Illusory sound shifts induced by the ventriloquist illusion evoke the mismatch negativity
Neuroscience Letters, 2004
The ventriloquist illusion arises when sounds are mislocated towards a synchronous but spatially discrepant visual event. Here, we investigated the ventriloquist illusion at a neurophysiological level. The question was whether an illusory shift in sound location was reflected in the auditory mismatch negativity (MMN). An 'oddball' paradigm was used whereby simultaneously presented sounds and flashes coming from the same location served as standard. The deviant consisted of a sound originating from the same source as the standard together with a flash at 208 spatial separation, which evoked an illusory sound shift. This illusory sound shift evoked an MMN closely resembling the MMN evoked by an actual sound shift. A visual-only control condition ruled out that the illusory-evoked MMN was confounded by the visual part of the audiovisual deviant. These results indicate that the crossmodal interaction on which the ventriloquist illusion is based takes place automatically at an early processing stage, within 200 ms after stimulus onset. q
Clinical Neurophysiology, 2002
The ventriloquism effect is the tendency to underestimate the spatial separation between synchronous auditory and visual signals moderately separated in space. If, as it is thought, this effect is pre-attentive, it could modulate the mismatch negativity (MMN) that indexes the automatic, pre-attentive detection of deviant auditory stimuli rarely occurring in a sequence of standard stimuli. We assessed the existence of an MMN evoked by auditory and visual signals made up of standard sounds coming from the same location as the visual signal and deviant sounds coming from lateral deviations (20 or 608). As first observed in a behavioral study, a ventriloquism effect occurred for 208 spatial separation but not for 608.
The aftereffects of ventriloquism: Generalization across sound-frequencies
Acta Psychologica, 2005
Exposure to synchronous but spatially discordant auditory and visual inputs produces, beyond immediate cross-modal biases, adaptive recalibrations of the respective localization processes that manifest themselves in aftereffects. Such recalibrations probably play an important role in maintaining the coherence of spatial representations across the various spatial senses. The present study is part of a research program focused on the way recalibrations generalize to stimulus values different from those used for adaptation. Considering the case of sound frequency, we recently found that, in contradiction with an earlier report, auditory aftereffects generalize nearly entirely across two octaves. In this new experiment, participants were adapted to an 18°auditory-visual discordance with either 400 or 6400 Hz tones, and their subsequent sound localization was tested across this whole four-octave frequency range. Substantial aftereffects, decreasing significantly with increasing difference between test and adapter frequency, were obtained at all combinations of adapter and test frequency. Implications of these results concerning the functional site at which visual recalibration of auditory localization might take place are discussed.
An fMRI Study of the Ventriloquism Effect
Cerebral Cortex, 2015
In spatial perception, visual information has higher acuity than auditory information and we often misperceive sound-source locations when spatially disparate visual stimuli are presented simultaneously. Ventriloquists make good use of this auditory illusion. In this study, we investigated neural substrates of the ventriloquism effect to understand the neural mechanism of multimodal integration. This study was performed in 2 steps. First, we investigated how sound locations were represented in the auditory cortex. Secondly, we investigated how simultaneous presentation of spatially disparate visual stimuli affects neural processing of sound locations. Based on the population rate code hypothesis that assumes monotonic sensitivity to sound azimuth across populations of broadly tuned neurons, we expected a monotonic increase of blood oxygenation leveldependent (BOLD) signals for more contralateral sounds. Consistent with this hypothesis, we found that BOLD signals in the posterior superior temporal gyrus increased monotonically as a function of sound azimuth. We also observed attenuation of the monotonic azimuthal sensitivity by spatially disparate visual stimuli. The alteration of the neural pattern was considered to reflect the neural mechanism of the ventriloquism effect. Our findings indicate that conflicting audiovisual spatial information of an event is associated with an attenuation of neural processing of auditory spatial localization.
Are the McGurk illusions affected by left or right presentation of the speaker face?
When conflicting syllables are presented in the auditory and visual modalities, two kinds of illusions have been reported by : combinations (cluster responses) and fusions (fused responses). In the present experiment, we examined the lateralization issue of these illusions. In one half of the session the center of the TV screen was displaced 5° to the right of straight ahead and in the other half, it was displaced 5° to the left. The sound, played at an average level of 40dB, always came from straight ahead. Neither for fusions nor for combinations did we found any difference in the percentage of illusion as a function of the hemifield in which the visual stimuli were presented. This result does not corroborate Diesch's (1995) finding. Using a somewhat different method, this author found a left hemifield advantage for fusions and a right hemifield advantage for combinations.
The visual mismatch negativity elicited with visual speech stimuli
Frontiers in Human Neuroscience, 2013
The visual mismatch negativity (vMMN), deriving from the brain's response to stimulus deviance, is thought to be generated by the cortex that represents the stimulus. The vMMN response to visual speech stimuli was used in a study of the lateralization of visual speech processing. Previous research suggested that the right posterior temporal cortex has specialization for processing simple non-speech face gestures, and the left posterior temporal cortex has specialization for processing visual speech gestures. Here, visual speech consonant-vowel (CV) stimuli with controlled perceptual dissimilarities were presented in an electroencephalography (EEG) vMMN paradigm. The vMMNs were obtained using the comparison of event-related potentials (ERPs) for separate CVs in their roles as deviant vs. their roles as standard. Four separate vMMN contrasts were tested, two with the perceptually far deviants (i.e., "zha" or "fa") and two with the near deviants (i.e., "zha" or "ta"). Only far deviants evoked the vMMN response over the left posterior temporal cortex. All four deviants evoked vMMNs over the right posterior temporal cortex. The results are interpreted as evidence that the left posterior temporal cortex represents speech contrasts that are perceived as different consonants, and the right posterior temporal cortex represents face gestures that may not be perceived as different CVs.
Directing spatial attention towards the illusory location of a ventriloquized sound
Acta Psychologica, 2001
In this study, we examined whether ventriloquism can rearrange external space on which spatial re¯exive attention operates. The task was to judge the elevation (up vs down) of auditory targets delivered in the left or the right periphery, taking no account of side of presentation. Targets were preceded by either auditory, visual, or audiovisual cues to that side. Auditory, but not visual cues had an eect on the speed of auditory target discrimination. On the other hand, a ventriloquized cue, consisting of a tone in central location synchronized with a light¯ash in the periphery, facilitated responses to targets appearing on the same side as thē ash. That eect presumably resulted from the attraction of the apparent location of the tone towards the¯ash, a well-known manifestation of ventriloquism. Ventriloquism thus can reorganize space in which re¯exive attention operates. Ó