Beyond perceptual modality: Auditory effects on visual perception (original) (raw)

Audition influences color processing in the sound-induced visual flash illusion

Multisensory interactions can lead to illusory percepts, as exemplified by the sound-induced extra flash illusion (SIFI: Shams, Kamitani, & Shimojo, 2000, 2002). In this illusion, an audiovisual stimulus sequence consisting of two pulsed sounds and a light flash presented within a 100 ms time window generates the visual percept of two flashes. Here, we used colored visual stimuli to investigate whether concurrent auditory stimuli can affect the perceived features of the illusory flash. Zero, one or two pulsed sounds were presented concurrently with either a red or green flash or with two flashes of different colors (red followed by green) in rapid sequence. By querying both the number and color of the participants' visual percepts, we found that the double flash illusion is stimulus specific: i.e., two sounds paired with one red or one green flash generated the percept of two red or two green flashes, respectively. This implies that the illusory second flash is induced at a level of visual processing after perceived color has been encoded. In addition, we found that the presence of two sounds influenced the integration of color information from two successive flashes. In the absence of any sounds, a red and a green flash presented in rapid succession fused to form a single orange percept, but when accompanied by two sounds, this integrated orange percept was perceived to flash twice on a significant proportion of trials. In addition, the number of concurrent auditory stimuli modified the degree to which the successive flashes were integrated to an orange percept vs. maintained as separate red-green percepts. Overall, these findings show that concurrent auditory input can affect both the temporal and featural properties of visual percepts.

Auditory Influences on Visual Temporal Rate Perception

Journal of Neurophysiology, 2002

Visual stimuli are known to influence the perception of auditory stimuli in spatial tasks, giving rise to the ventriloquism effect. These influences can persist in the absence of visual input following a period of exposure to spatially disparate auditory and visual stimuli, a phenomenon termed the ventriloquism aftereffect. It has been speculated that the visual dominance over audition in spatial tasks is due to the superior spatial acuity of vision compared with audition. If that is the case, then the auditory system should dominate visual perception in a manner analogous to the ventriloquism effect and aftereffect if one uses a task in which the auditory system has superior acuity. To test this prediction, the interactions of visual and auditory stimuli were measured in a temporally based task in normal human subjects. The results show that the auditory system has a pronounced influence on visual temporal rate perception. This influence was independent of the spatial location, spe...

Auditory manipulation of visual perception

2008

Psychological research on cross-modal auditory-visual perception has focused predominantly on the manipulation of sensory information by visual information. There are relatively few studies of the way auditory stimuli may affect other sensory information. The Sound-induced Illusory Flash is one illusory paradigm that involves the auditory system biasing visual information. However, little is known about this cross-modal illusion. More research is needed into the structure of the illusion that investigates the different conditions under which the Soundinduced Illusory Flash manifests and is enhanced or reduced. The research conducted for this thesis investigates the effect of new auditory stimulus variables on the Sound-induced Illusory Flash. The variables to be discussed concern the formation of a contrast in the auditory stimuli, with the contrast creating a rhythm that emphasises the discontinuous nature of the auditory stimuli, and therefore emphasises the illusory percept. The auditory stimulus contrasts include pitch separation with the octave interval, using the frequencies of 261.5 and 523Hz; and spatial separation in the auditory stimuli, presenting the monophonic auditory stimuli binaurally so that individual tones alternate between the left and right channels.

Auditory motion affects visual motion perception in a speeded discrimination task

Experimental Brain Research, 2007

Transient auditory stimuli have been shown to influence the perception of ambiguous 2D visual motion displays (the bouncing-disks effect; e.g. Sekuler et al. in Nature 385:308, 1997). The question addressed here was whether continuous moving auditory stimuli can also influence visual motion perception under the same experimental conditions. In Experiment 1, we used a modification of Sanabria et al.’s (Exp Brain Res 157:537–541, 2004) paradigm (involving an indirect behavioural measure of the bouncing-disks effect), in which the 2D visual display was presented together with either a brief tone, a continuous moving sound, or in the absence of any form of auditory stimulation. Crucially, the results showed that, together with the effect of the brief tone on bouncing trials, the presence of the continuous moving sound speeded-up participants’ responses on streaming trials as compared to the brief tone or no sound conditions. The results of a second experiment revealed that the effect of the continuous moving sound reported in Experiment 1 was not caused simply by the presence of continuous auditory stimulation per se.

Sound Frequency and Aural Selectivity in Sound-Contingent Visual Motion Aftereffect

PLoS ONE, 2012

Background: One possible strategy to evaluate whether signals in different modalities originate from a common external event or object is to form associations between inputs from different senses. This strategy would be quite effective because signals in different modalities from a common external event would then be aligned spatially and temporally. Indeed, it has been demonstrated that after adaptation to visual apparent motion paired with alternating auditory tones, the tones begin to trigger illusory motion perception to a static visual stimulus, where the perceived direction of visual lateral motion depends on the order in which the tones are replayed. The mechanisms underlying this phenomenon remain unclear. One important approach to understanding the mechanisms is to examine whether the effect has some selectivity in auditory processing. However, it has not yet been determined whether this aftereffect can be transferred across sound frequencies and between ears.

A single auditory tone alters the perception of multiple visual events

Journal of vision, 2014

We aimed to show that a single auditory tone crossmodally affects multiple visual events using a multiple stream/bounce display (SBD), consisting of two disk pairs moving toward each other at equal speeds, coinciding, and then moving apart in a two-dimensional (2-D) display. The temporal offsets were manipulated between the coincidences of the disk pairs (0 to ±240 ms) by staggering motion onset between the pairs. A tone was presented at the coincidence timing of one of the disk pairs on half of the trials. Participants judged whether the disks in each of two pairs appeared to stream through or bounce off each other. Results show that a tone presented at either of the disk pairs' coincidence points promoted bouncing percepts in both disk pairs compared to no-tone trials. Perceived bouncing persisted in the disk-pair whose coincidence was offset 60 ms before and up to more than 120 ms after the audiovisual coincidence timing of the other disk-pair. The temporal window of bounce p...

Sound enhances visual perception: Cross-modal effects of auditory organization on vision

Journal of Experimental Psychology-human Perception and Performance, 2000

Six experiments demonstrated cross-modal influences from the auditory on the visual modality at an early level of perceptual organization. Participants had to detect a visual target in a rapidly changing sequence of visual distractors. A high tone embedded in a sequence of low tones improved detection of a synchronously presented visual target (Exp. 1), but the effect disappeared when the high tone was presented before the target (Exp 2). Rhythmically-based or order-based anticipation was unlikely to account for the effect because the improvement was unaffected by whether there was jitter (Exp 3 ), or a random number of distractors between successive targets (Exp 4). The facilitatory effect was greatly reduced when the tone was less abrupt and part of a melody (Exps 5 and 6). These results show that perceptual organization in the auditory modality can have an effect on perceptibility in the visual modality.

A deafening flash! Visual interference of auditory signal detection

2017

In some people, visual stimulation evokes auditory sensations. How prevalent and how perceptually real is this? 22% responded 'Yes' when asked whether they heard faint sounds accompanying flash stimuli, and showed significantly better ability to discriminate visual ‘Morse-code’ sequences. This benefit might arise from an ability to recode visual signals as sounds, thus taking advantage of superior temporal acuity of audition. In support of this, those who showed better visual relative to auditory sequence discrimination also had poorer auditory detection in the presence of uninformative visual flashes, though this was independent of awareness of visually-evoked sounds. Thus a visually-evoked auditory representation may occur subliminally and disrupt detection of real auditory signals. The frequent natural correlation between visual and auditory stimuli might explain the surprising prevalence of this phenomenon. Overall, our results suggest that learned correspondences between strongly correlated modalities may provide a precursor for some synaesthetic abilities.

Visual illusion induced by sound

2002

We present the first cross-modal modification of visual perception which involves a phenomenological change in the quality-as opposed to a small, gradual, or quantitative change-of the percept of a non-ambiguous visual stimulus. We report a visual illusion which is induced by sound: when a single flash of light is accompanied by multiple auditory beeps, the single flash is perceived as multiple flashes. We present two experiments as well as several observations which establish that this alteration of the visual percept is due to cross-modal perceptual interactions as opposed to cognitive, attentional, or other origins. The results of the second experiment also reveal that the temporal window of these audiovisual interactions is approximately 100 ms.

Alternation of Sound Location Induces Visual Motion Perception of a Static Object

PLoS ONE, 2009

Background: Audition provides important cues with regard to stimulus motion although vision may provide the most salient information. It has been reported that a sound of fixed intensity tends to be judged as decreasing in intensity after adaptation to looming visual stimuli or as increasing in intensity after adaptation to receding visual stimuli. This audiovisual interaction in motion aftereffects indicates that there are multimodal contributions to motion perception at early levels of sensory processing. However, there has been no report that sounds can induce the perception of visual motion.