Illusory sound shifts induced by the ventriloquist illusion evoke the mismatch negativity (original) (raw)
Related papers
Clinical Neurophysiology, 2002
The ventriloquism effect is the tendency to underestimate the spatial separation between synchronous auditory and visual signals moderately separated in space. If, as it is thought, this effect is pre-attentive, it could modulate the mismatch negativity (MMN) that indexes the automatic, pre-attentive detection of deviant auditory stimuli rarely occurring in a sequence of standard stimuli. We assessed the existence of an MMN evoked by auditory and visual signals made up of standard sounds coming from the same location as the visual signal and deviant sounds coming from lateral deviations (20 or 608). As first observed in a behavioral study, a ventriloquism effect occurred for 208 spatial separation but not for 608.
Experimental brain research, 2016
Visual capture and the ventriloquism aftereffect resolve spatial disparities of incongruent auditory visual (AV) objects by shifting auditory spatial perception to align with vision. Here, we demonstrated the distinct temporal characteristics of visual capture and the ventriloquism aftereffect in response to brief AV disparities. In a set of experiments, subjects localized either the auditory component of AV targets (A within AV) or a second sound presented at varying delays (1-20 s) after AV exposure (A2 after AV). AV targets were trains of brief presentations (1 or 20), covering a ±30° azimuthal range, and with ±8° (R or L) disparity. We found that the magnitude of visual capture generally reached its peak within a single AV pair and did not dissipate with time, while the ventriloquism aftereffect accumulated with repetitions of AV pairs and dissipated with time. Additionally, the magnitude of the auditory shift induced by each phenomenon was uncorrelated across listeners and visu...
The ventriloquism illusion does modulate the Mismatch Negativity (MMN)
Audiovisual interactions observed in the ventriloquism and McGurk illusions were compared, using the same materials. The ventriloquism effect was estimated through a discordance detection task and the McGurk illusion through an identification task. The stimuli were visually displayed on a screen located in front of the participants' head and acoustically delivered through one of nine hidden loudspeakers located from straight ahead (0°) to 80° (20° steps) left and right. The speaker's face was either upright or inverted. The ventriloquism effect was affected by the degree of spatial separation, but unaffected by upright vs. inverted presentation of the face, or by the congruency of the stimuli. The McGurk illusion was of the same size whatever the loudspeaker location but was reduced by face inversion. The differences in the spatial and cognitive rules that govern both interactions are discussed in terms of specific functionality of the underlying mechanisms.
The aftereffects of ventriloquism: Generalization across sound-frequencies
Acta Psychologica, 2005
Exposure to synchronous but spatially discordant auditory and visual inputs produces, beyond immediate cross-modal biases, adaptive recalibrations of the respective localization processes that manifest themselves in aftereffects. Such recalibrations probably play an important role in maintaining the coherence of spatial representations across the various spatial senses. The present study is part of a research program focused on the way recalibrations generalize to stimulus values different from those used for adaptation. Considering the case of sound frequency, we recently found that, in contradiction with an earlier report, auditory aftereffects generalize nearly entirely across two octaves. In this new experiment, participants were adapted to an 18°auditory-visual discordance with either 400 or 6400 Hz tones, and their subsequent sound localization was tested across this whole four-octave frequency range. Substantial aftereffects, decreasing significantly with increasing difference between test and adapter frequency, were obtained at all combinations of adapter and test frequency. Implications of these results concerning the functional site at which visual recalibration of auditory localization might take place are discussed.
Seeing and Perceiving, 2012
Exposure to synchronous but spatially discordant auditory and visual inputs produces adaptive recalibration of the respective localization processes, which manifest themselves in measurable aftereffects. Here we report two experiments that examined the time course of visual recalibration of apparent sound location in order to establish the build-up and dissipation of recalibration. In Experiment 1 participants performed a sound localization task before and during exposure to an auditory-visual discrepancy. In Experiment 2, participants performed a sound localization task before and after 60, 180, or 300 exposures to the discrepancy and aftereffects were measured across a series of post-adaptation sound localization trials. The results show that recalibration is very fast. Substantial aftereffects are obtained after only 18-24 exposures and asymptote appears to be reached between 60 and 180 exposures. The rate of adaptation was independent of the size of the discrepancy. The retention of the aftereffect was strong, as we found no dissipation, not even after as few as 60 exposure trials.
The impact of spatial incongruence on an auditory-visual illusion
PloS one, 2009
Background: The sound-induced flash illusion is an auditory-visual illusion -when a single flash is presented along with two or more beeps, observers report seeing two or more flashes. Previous research has shown that the illusion gradually disappears as the temporal delay between auditory and visual stimuli increases, suggesting that the illusion is consistent with existing temporal rules of neural activation in the superior colliculus to multisensory stimuli. However little is known about the effect of spatial incongruence, and whether the illusion follows the corresponding spatial rule. If the illusion occurs less strongly when auditory and visual stimuli are separated, then integrative processes supporting the illusion must be strongly dependant on spatial congruence. In this case, the illusion would be consistent with both the spatial and temporal rules describing response properties of multisensory neurons in the superior colliculus.
Directing spatial attention towards the illusory location of a ventriloquized sound
Acta Psychologica, 2001
In this study, we examined whether ventriloquism can rearrange external space on which spatial re¯exive attention operates. The task was to judge the elevation (up vs down) of auditory targets delivered in the left or the right periphery, taking no account of side of presentation. Targets were preceded by either auditory, visual, or audiovisual cues to that side. Auditory, but not visual cues had an eect on the speed of auditory target discrimination. On the other hand, a ventriloquized cue, consisting of a tone in central location synchronized with a light¯ash in the periphery, facilitated responses to targets appearing on the same side as thē ash. That eect presumably resulted from the attraction of the apparent location of the tone towards the¯ash, a well-known manifestation of ventriloquism. Ventriloquism thus can reorganize space in which re¯exive attention operates. Ó
On perceived synchrony—neural dynamics of audiovisual illusions and suppressions
Brain Research, 2008
Whenever temporally incongruent audiovisual sequences are presented, the perceived flash rate follows the physical flutter rate. Increasing the auditory flutter rate increases the perceived flicker rate (visual illusions). Likewise, decreasing the flutter rate decreases the perceived flicker rate (visual suppressions). Here, we investigated the electrophysiological correlates of this perceptual phenomenon. Two sequences of visual flashes and auditory beeps were presented either synchronously (both visual flashes (F) and auditory beeps (B) at 3 or 5 Hz, respectively) or asynchronously at different rates (3F5B or 5F3B). Event-related potentials were acquired, while subjects reported the perceived number of flashes (response options: 3, 4, and 5). During asynchronous trials, subjects' flash counts were significantly higher when the flutter rate exceeded the flicker rate (i.e. visual illusions occurred); and lower flutter rate was below the flicker rate (i.e. visual suppressions occurred). Differential brain responses for reported illusions and suppressions (incorrect flash counts) vs. no-illusions/ suppressions (correct flash counts) were found over parieto-occipital sites, followed by slow modulations over frontal and occipital areas. Importantly, the modulation over occipital electrodes starting around 500 ms had an inverse polarity for illusions vs. suppressions.