Attentional modulations of audiovisual interactions in apparent motion: Temporal ventriloquism effects on perceived visual speed (original) (raw)

Auditory Influences on Visual Temporal Rate Perception

Journal of Neurophysiology, 2002

Visual stimuli are known to influence the perception of auditory stimuli in spatial tasks, giving rise to the ventriloquism effect. These influences can persist in the absence of visual input following a period of exposure to spatially disparate auditory and visual stimuli, a phenomenon termed the ventriloquism aftereffect. It has been speculated that the visual dominance over audition in spatial tasks is due to the superior spatial acuity of vision compared with audition. If that is the case, then the auditory system should dominate visual perception in a manner analogous to the ventriloquism effect and aftereffect if one uses a task in which the auditory system has superior acuity. To test this prediction, the interactions of visual and auditory stimuli were measured in a temporally based task in normal human subjects. The results show that the auditory system has a pronounced influence on visual temporal rate perception. This influence was independent of the spatial location, spe...

Temporal Ventriloquism: Sound Modulates the Flash-Lag Effect

Journal of Experimental Psychology-human Perception and Performance, 2004

A sound presented in close temporal proximity to a visual stimulus can alter the perceived temporal dimensions of the visual stimulus (temporal ventriloquism). In this article, the authors demonstrate temporal ventriloquism in the flash-lag effect (FLE), a visual illusion in which a flash appears to lag relative to a moving object. In Experiment 1, the magnitude and the variability of the FLE were reduced, relative to a silent condition, when a noise burst was synchronized with the flash. In Experiment 2, the sound was presented before, at, or after the flash (Ϯϳ100 ms), and the size of the FLE varied linearly with the delay of the sound. These findings demonstrate that an isolated sound can sharpen the temporal boundaries of a flash and attract its temporal occurrence.

Spatial Attention and Audiovisual Interactions in Apparent Motion

Journal of Experimental Psychology-human Perception and Performance, 2007

In this study, the authors combined the cross-modal dynamic capture task (involving the horizontal apparent movement of visual and auditory stimuli) with spatial cuing in the vertical dimension to investigate the role of spatial attention in cross-modal interactions during motion perception. Spatial attention was manipulated endogenously, either by means of a blocked design or by predictive peripheral cues, and exogenously by means of nonpredictive peripheral cues. The results of 3 experiments demonstrate a reduction in the magnitude of the cross-modal dynamic capture effect on cued trials compared with uncued trials. The introduction of neutral cues (Experiments 4 and 5) confirmed the existence of both attentional costs and benefits. This attention-related reduction in cross-modal dynamic capture was larger when a peripheral cue was used compared with when attention was oriented in a purely endogenous manner. In sum, the results suggest that spatial attention reduces illusory binding by facilitating the segregation of unimodal signals, thereby modulating audiovisual interactions in information processing. Thus, the effect of spatial attention occurs prior to or at the same time as cross-modal interactions involving motion information.

Auditory motion affects visual motion perception in a speeded discrimination task

Experimental Brain Research, 2007

Transient auditory stimuli have been shown to influence the perception of ambiguous 2D visual motion displays (the bouncing-disks effect; e.g. Sekuler et al. in Nature 385:308, 1997). The question addressed here was whether continuous moving auditory stimuli can also influence visual motion perception under the same experimental conditions. In Experiment 1, we used a modification of Sanabria et al.’s (Exp Brain Res 157:537–541, 2004) paradigm (involving an indirect behavioural measure of the bouncing-disks effect), in which the 2D visual display was presented together with either a brief tone, a continuous moving sound, or in the absence of any form of auditory stimulation. Crucially, the results showed that, together with the effect of the brief tone on bouncing trials, the presence of the continuous moving sound speeded-up participants’ responses on streaming trials as compared to the brief tone or no sound conditions. The results of a second experiment revealed that the effect of the continuous moving sound reported in Experiment 1 was not caused simply by the presence of continuous auditory stimulation per se.

Auditory temporal modulation of the visual Ternus effect: the influence of time interval

Experimental Brain Research, 2010

Research on multisensory interactions has shown that the perceived timing of a visual event can be captured by a temporally proximal sound. This effect has been termed 'temporal ventriloquism effect.' Using the Ternus display, we systematically investigated how auditory configurations modulate the visual apparent-motion percepts. The Ternus display involves a multielement stimulus that can induce either of two different percepts of apparent motion: 'element motion' or 'group motion'. We found that two sounds presented in temporal proximity to, or synchronously with, the two visual frames, respectively, can shift the transitional threshold for visual apparent motion . However, such effects were not evident with single-sound configurations (Experiment 2). A further experiment (Experiment 4) provided evidence that time interval information is an important factor for crossmodal interaction of audiovisual Ternus effect. The auditory interval was perceived as longer than the same physical visual interval in the sub-second range. Furthermore, the perceived audiovisual interval could be predicted by optimal integration of the visual and auditory intervals.

Direction of Visual Apparent Motion Driven Solely by Timing of a Static Sound

Current Biology, 2008

In temporal ventriloquism, auditory events can illusorily attract perceived timing of a visual onset . We investigated whether timing of a static sound can also influence spatio-temporal processing of visual apparent motion, induced here by visual bars alternating between opposite hemifields. Perceived direction typically depends on the relative interval in timing between visual left-right and right-left flashes (e.g., rightwards motion dominating when leftto-right interflash intervals are shortest [4]). In our new multisensory condition, interflash intervals were equal, but auditory beeps could slightly lag the right flash, yet slightly lead the left flash, or vice versa. This auditory timing strongly influenced perceived visual motion direction, despite providing no spatial auditory motion signal whatsoever. Moreover, prolonged adaptation to such auditorily driven apparent motion produced a robust visual motion aftereffect in the opposite direction, when measured in subsequent silence. Control experiments argued against accounts in terms of possible auditory grouping, or possible attention capture. We suggest that the motion arises because the sounds change perceived visual timing, as we separately confirmed. Our results provide a new demonstration of multisensory influences on sensory-specific perception , with timing of a static sound influencing spatio-temporal processing of visual motion direction.

The effect of timing and spatial separation on the velocity of auditory apparent motion

Perception & Psychophysics, 1998

Previously, it was shown that the minimum conditions for the illusion of auditory apparent motion (AAM) depend on stimulus timing hut not spatial separation. In the present experiment, the effects of stimulus timing and source separation on the perceived velocity of AAM were examined. Eight listeners estimated the velocity, duration, and distance traveled of AAM, using a no-modulus, magnitude estimation procedure. Four burst durations (25, 50,100, and 300 msec), 10stimulus onset asynchronies (SOAs; 30, 40, 50, 60, 70,80, 90, 100,110, and 120msec) and two separations (10°and 40°)were tested. Perceived velocity estimates were related to the total duration (burst duration + SOA) ofthe stimulus sequence. The effect of separation on velocity was extremely small but statistically significant. These results are similar to those obtained previously on the minimum conditions for AAM. Duration estimates were related only to total duration, but separation estimates were related to both separation and total duration. These results suggest that velocity is possibly a primary dimension of AAM that is independent of source separation.

Low-Level Visual Processing Speed Modulates Judgment of Audio-Visual Simultaneity

Interdisciplinary Information Sciences, 2015

Temporal consistency between visual and auditory presentations is necessary for integration of visual and auditory information. Subjective simultaneity perception is more important than the synchrony of physical inputs for temporal consistency. Our previous studies have shown that audio-visual integration is difficult even if the visual and auditory inputs are physically synchronous when visual processing is slow. In the present study, we examined the effects of visual processing speed on audio-visual integration using a simultaneity judgment task. Visual processing speed was manipulated by varying the spatial frequency of visual stimuli. High spatial frequency stimuli require a longer processing time because visual responses to high spatial frequencies are slow. The results indicated that the difference between subjective and physical synchrony was larger in high spatial frequency than in low spatial frequency. Thus, the spatial frequency of the visual stimulus affected the judgments of simultaneity for visual and auditory stimuli. The effects of visual processing speed on audio-visual integration are believed to occur at a lower-order stage of sensory processing.

Auditory and visual attention-based apparent motion share functional parallels

Perception & Psychophysics, 2008

A perception of coherent motion can be obtained in an otherwise ambiguous or illusory visual display by directing one's attention to a feature and tracking it. We demonstrate an analogous auditory effect in two separate sets of experiments. The temporal dynamics associated with the attention-dependent auditory motion closely matched those previously reported for attention-based visual motion. Since attention-based motion mechanisms appear to exist in both modalities, we also tested for multimodal (audiovisual) attention-based motion, using stimuli composed of interleaved visual and auditory cues. Although subjects were able to track a trajectory using cues from both modalities, no one spontaneously perceived "multimodal motion" across both visual and auditory cues. Rather, they reported motion perception only within each modality, thereby revealing a spatiotemporal limit on putative cross-modal motion integration. Together, results from these experiments demonstrate the existence of attention-based motion in audition, extending current theories of attention-based mechanisms from visual to auditory systems.

Assessing the effect of visual and tactile distractors on the perception of auditory apparent motion

Experimental Brain Research, 2005

In this study we investigated the effect of the directional congruency of tactile, visual, or bimodal visuotactile apparent motion distractors on the perception of auditory apparent motion. Participants had to judge the direction in which an auditory apparent motion stream moved (left-to-right or right-to-left) while trying to ignore one of a range of distractor stimuli, including unimodal tactile or visual, bimodal visuotactile, and crossmodal (i.e., composed of one visual and one tactile stimulus) distractors. Significant crossmodal dynamic capture effects (i.e., better performance when the target and distractor stimuli moved in the same direction rather than in opposite directions) were demonstrated in all conditions. Bimodal distractors elicited more crossmodal dynamic capture than unimodal distractors, thus providing the first empirical demonstration of the effect of information presented simultaneously in two irrelevant sensory modalities on the perception of motion in a third (target) sensory modality. The results of a second experiment demonstrated that the capture effect reported in the crossmodal distractor condition was most probably attributable to the combined effect of the individual static distractors (i.e., to ventriloquism) rather than to any emergent property of crossmodal apparent motion.