Middle Temporal Visual Area Microstimulation Influences Veridical Judgments of Motion Direction (original) (raw)

ARTICLE, Behavioral/Systems

Journal of Neuroscience 1 November 2002, 22 (21) 9530-9540; https://doi.org/10.1523/JNEUROSCI.22-21-09530.2002

Loading

Abstract

Microstimulation of direction columns in the middle temporal visual area (MT, or V5) provides a powerful tool for probing the relationship between cortical physiology and visual motion perception. In the current study we obtained “veridical” reports of perceived motion from rhesus monkeys by permitting a continuous range of possible responses that mapped isomorphically onto a continuous range of possible motion directions. In contrast to previous studies, therefore, the animals were freed from experimenter-imposed “categories” that typify forced choice tasks. We report three new findings: (1) MT neurons with widely disparate preferred directions can cooperate to shape direction estimates, inconsistent with a pure “winner-take-all” read-out algorithm and consistent with a distributed coding scheme like vector averaging, whereas neurons with nearly opposite preferred directions seem to compete in a manner consistent with the winner-take-all hypothesis, (2) microstimulation can influence direction estimates even when paired with the most powerful motion stimuli available, and (3) microstimulation effects can be elicited when a manual response (instead of our standard oculomotor response) is used to communicate the perceptual report.

Animals depend for their survival on an accurate internal representation of the external environment. The visual system contributes to this representation by reconstructing the visual scene from the retinal image. The first step in scene reconstruction involves encoding local quantities in the image, like the orientation of line segments or the direction of motion of texture elements. Notably, however, even simple visual stimuli activate broad populations of cortical neurons with diverse stimulus preferences and highly variable responses. For example, a large majority of neurons in monkey middle temporal visual area (MT) respond selectively to the direction of stimulus motion (Zeki, 1974; Maunsell and Van Essen, 1983), and these neurons are organized into cortical columns on the basis of their preferred direction of motion (Albright et al., 1984). MT neurons are quite broadly tuned for direction, however (Albright, 1984), so that even the simplest motion stimulus will elicit a broad profile of activity across MT (Fig.1A). Thus, the visual system faces a fundamental challenge in deriving precise representations of sensory variables from seemingly imprecise neuronal responses. How do monkeys extract accurate estimates of motion direction from neural activity in MT?

Fig. 1.

Fig. 1.

Schematic depiction of neuronal activation profiles in cortical area MT. A, A patch of cortex illustrating the topographic representation of preferred direction (arrows) across MT. The bell-shaped curve_represents a hypothetical activation profile in response to a directional visual stimulus. The height of the curve indicates the level of discharge. Because MT neurons are broadly tuned for direction, neurons with a range of preferred directions become active. The_shaded region indicates the most active neurons in the population. B, In addition to the visually induced activity, a second, electrically evoked profile of activity has been added. Note that the shape and extent of this profile is hypothetical. The dashed curves represent the hypothetical profiles of activation elicited by the visual stimulus or microstimulation alone, whereas the solid curve represents the sum of the two.

Several algorithms have been proposed that could reconstruct sensory variables like direction from patterns of discharge in neural populations (Adelson and Movshon, 1982; Georgopoulos et al., 1986;Steinmetz et al., 1987; Lee et al., 1988; Deneve et al., 1999). One approach, winner-take-all, involves locating the column of neurons most activated by the stimulus (Fig. 1A, shaded region) and assigning their preferred direction of motion to the stimulus. According to this idea, less active neurons make no contribution to a monkey's direction percept. In a competing approach, vector averaging, every active neuron “votes” for its preferred direction, and the average of these directions is assigned to the visual stimulus. Vector averaging is one member of a family of distributed coding schemes in which a sensory (or motor) variable is represented by a distribution of activity across neurons with different stimulus preferences. In this paper, vector averaging will serve as proxy for this family of distributed codes, all of which stand in contrast to the winner-take-all approach.

Competing models like these can be tested by perturbing artificially the profile of activation in MT and measuring the impact on behavior. In two previous studies, we used electrical microstimulation to perturb MT activation while monkeys performed very different behavioral tasks: perceptual discrimination versus initiation of pursuit eye movements (Salzman and Newsome, 1994; Groh et al., 1997). Because the two studies yielded seemingly discrepant conclusions, we sought to resolve the issue in the current study by using a perceptual paradigm that combines elements of the two previous paradigms. We find that MT neurons with a wide range of preferred directions can cooperate to influence an animal's estimate of motion direction, consistent with a distributed representation like vector averaging. Neurons with nearly opposite direction preferences, however, seem to compete to determine a monkey's direction percept, consistent with a winner-take-all regime. Thus, more than one mechanism may contribute toward interpretation of activity profiles in MT.

MATERIALS AND METHODS

Our methods for surgical preparation, animal training, electrophysiological recording, and microstimulation in rhesus monkeys (Macaca mulatta) are described in detail elsewhere (Britten et al., 1992; Salzman et al., 1992). All procedures were approved by the Stanford University Animal Care and Use Committee.

Visual stimuli

Visual stimuli were generated by a personal computer (486) using a Cambridge Research graphics board (VSG 2/3) and displayed on a monitor placed 57 cm away from the monkey. The random dot stimuli have been described in detail previously (Britten et al., 1992). Briefly, the stimulus consisted of white dots on a black background. Each dot was displayed for one frame and then replotted 16.7 msec later either at an appropriate spatial displacement for the desired speed and direction of the global stimulus motion or at a random location. The probability that a dot would be displaced rather than plotted at a random location corresponds to the “coherence” of the visual stimulus, the percentage of dots that move in the same direction. So, at 100% coherence, all dots move with the same speed and direction, and at 0% coherence, all dots move randomly. The size of each dot was ∼0.1°; dot density was 32 dots/° per second; dots had a luminance of 24.3 cd/m2.

Electrophysiology

Neuronal activity was recorded using parylene-coated tungsten microelectrodes (Microprobe, Bethesda, MD; impedance, 0.5–1.0 MΩ at 1 kHz). Electrodes were inserted into the occipital lobe through stainless steel guide tubes held in place by a plastic grid attached to the inside of the recording cylinder (Crist et al., 1988). Our angle of approach was ∼20° above the horizontal plane, beginning in striate cortex, passing across the lunate sulcus, and then onto the posterior bank of the superior temporal sulcus. MT was identified in preliminary mapping experiments based on its high percentage of direction-selective units, its characteristic topography, and the stereotyped pattern of gray matter, white matter, and sulci along the electrode tracks.

For multiunit recordings, an “event” was considered to be any excursion of the recorded voltage trace above a set threshold (this might correspond to an action potential from a single neuron or a signal from several superimposed spikes). The absolute frequency of multiunit responses is somewhat arbitrary, depending on the threshold level set by the experimenter. We reduced the arbitrary nature of the measurement by fixing the filter bandpass on our amplifier (0.5–5 kHz) and by adjusting the threshold level for each site to obtain a spontaneous activity level in the range from 50–150 events/sec. This multiunit measurement is likely to reflect the summed spiking activity of several neurons near the tip of the recording electrode.

Microstimulation was applied through the tungsten microelectrodes described above. We microstimulated selected sites in MT using a pulse generator in series with a stimulus isolation unit. The stimulating pulses were biphasic, cathodal pulse leading. The cathodal and anodal pulses were each 0.2 msec in duration, with a 0.1 msec pause between the two phases. The frequency of stimulation was 200 Hz, and the current was 10 μA for monkeys R and G, and 25 μA for monkey H.

Our goal in these experiments was to activate selectively neurons that had a similar preferred direction. We therefore attempted to place the electrode in the middle of a cluster of neurons having similar physiological properties. The microstimulation parameters were selected to restrict current spread to neurons near the stimulation site. We selected sites in which neurons encountered over at least 200–300 μm of electrode travel had similar preferred directions, preferred speeds, and receptive field locations. We then positioned the tip of the electrode at the midpoint of the site.

Experimental protocol and task design

Once a stimulation site was selected, monkeys performed three different blocks of trials in each experimental session.

Multiunit direction tuning block. The first block of trials involved a passive fixation task during which the direction tuning of multiunit activity was measured. A trial began when the monkey acquired a fixation spot on the display monitor. He was then required to maintain fixation within a 3 × 3° electronic window for the remainder of the trial. Once the monkey began to fixate, a random dot motion stimulus was presented at 100% coherence moving in one of eight possible directions spaced at 45° intervals spanning 360°. Each stimulus was presented four times for a duration of 1.5 sec. The stimulus was placed inside the receptive field of neurons at the stimulation site, and the speed of the stimulus was matched to their preferred speed.

Direction identification block. The direction identification task used in this study gave the monkey complete freedom to report veridically the perceived direction of motion in the visual stimulus. In contrast, previous psychophysical paradigms used in this laboratory required animals to make directional choices in the form of arbitrary categories, or “bins” defined by the experimenter.

Once the direction tuning of the stimulation site was measured, the monkey performed a direction identification task (Fig.2A). The monkey initiated each trial by acquiring a fixation point. After a 600 msec delay, a random dot motion stimulus appeared, surrounded by a thin annulus (the “target ring”). The motion stimulus was presented for 1 sec while the monkey continued to fixate. The motion stimulus and the fixation point were then extinguished while the target ring remained illuminated. The animal reported his estimate of the direction of motion in the stimulus by making a saccade to the location on the target ring toward which the dots had moved. Saccade endpoint was defined as the eye position after the primary saccade. Saccade initiation and termination were measured objectively using a velocity criterion of 20°/sec. If the saccade fell within an electronically defined window around the correct location on the target ring, the monkey received a liquid reward. The size of the reward window was typically ∼5 × 5° of visual angle (corresponding to ∼50° of angle around the target ring), but was adjusted modestly from experiment to experiment depending on the animal's performance under the specific conditions of each experiment. The same window size was always used for both stimulated and nonstimulated trials within a block, and the location of the window always depended only on the visual direction, regardless of whether microstimulation was applied. This reward protocol resulted in a modestly lower reward rate on trials when microstimulation was applied than on trials when only the visual stimulus was presented (73 vs 82%).

Fig. 2.

Fig. 2.

Psychophysical task and behavioral performance.A, Schematic illustration of the direction identification task. The three panels depict three key epochs in each trial. A trial begins when the monkey looks at a fixation point (FP) on the display monitor (left panel). Converging dashed lines indicate the monkey's direction of gaze. The _dashed circle_represents the receptive field (RF) of the MT column under study (not visible to the monkey). After a variable delay, a random dot motion stimulus appears moving in one of 8 or 18 possible directions (middle panel). The dot stimulus appears inside an annulus, or target ring. The motion stimulus is presented for 1 sec while the animal maintains fixation. The motion stimulus and the fixation point are then extinguished, and the animal makes a saccadic eye movement to the location on the target ring toward which the dots had moved (right panel).B, Saccade endpoints for one block of trials. Each black point represents the endpoint of a saccade. In this block, 18 different directions were presented spanning 360° in equal intervals. The directions were randomly interleaved, and each was presented 20 times. The coherence of the visual stimulus was 80%. No microstimulation was applied on these trials. The gray square represents the dimensions of the electronically defined reward window (not visible to the monkey) in this block of trials for one direction of visual motion.C, Mean and circular SD of the monkey's direction estimates plotted as a function of the direction of the visual stimulus. The diagonal represents perfect performance of the task.

The coherence of the visual stimulus was held constant for each block of trials and ranged from 25 to 100% across experiments. On each trial, the direction of stimulus motion was selected randomly from a range of possible directions. In some experiments (“large block”), there were 18 possible directions uniformly spaced at 20° intervals and spanning 360°. In other experiments (“small block”), there were eight possible directions spanning 360° in 45° increments. On 50% of trials (“interaction trials”), chosen randomly, electrical stimulation was applied to MT; the stimulation train precisely coincided in time with the presentation of the visual stimulus. Thus, there were 36 different conditions in large blocks and 16 different conditions in small blocks.

We chose other parameters of the visual stimulus (location, size, and speed of motion) to maximize the multiunit response of the MT column under study. This task differs from previous perceptual tasks (Britten et al., 1992; Celebrini and Newsome, 1994; Salzman and Newsome, 1994) in that the animal views relatively high coherence motion stimuli and provides a veridical estimate of the direction of motion, unconstrained by experimenter imposed categories.

Zero percent coherence block. After each direction identification block, the monkeys performed a second block of identification trials with the following difference. Instead of using a single, relatively high coherence motion stimulus for all trials, 0% coherence trials were randomly interleaved with two other low coherences (e.g., 12 and 24%). Otherwise the task was the same. Electrical stimulation was applied on half of 0% coherence trials, chosen randomly. Monkeys were rewarded on a random 25% of 0% coherence trials and were rewarded based on the visual direction for the higher coherence trials. By comparing a monkey's direction estimates on 0% coherence trials with and without microstimulation, we could assess the electrically induced direction signal in the absence of a coherent visual direction signal. This provided a second measure (in addition to the multiunit direction tuning) of the directional signal introduced by microstimulation.

Manual response task

One monkey was trained on a variant of the direction identification task described above (see Fig. 6A). The manual response version of the task differs only in the method the monkey used to report his direction estimate. Instead of making a saccade to the target ring, the monkey positioned a cursor around the circumference of the stimulus aperture. The final cursor position on each trial indicated the direction of motion in the stimulus, just as the location of the saccade endpoint did in the saccade version of the task. The monkey positioned the cursor by moving a manipulandum in a circular track (one degree of freedom). The steps of the task on each trial were as follows. An initial target appeared in a random location around the circumference of the stimulus aperture. The monkey was required to align the cursor with this target. Once he had aligned the cursor, a fixation point appeared, and the monkey acquired the fixation point. After a 600 msec delay, the motion stimulus appeared for 1 sec while the monkey continued to fixate and maintain the cursor in its initial position. The motion stimulus, the fixation point, and the initial target were then extinguished, and the monkey was free to move the cursor around the circumference of the stimulus aperture to the location corresponding to the direction of motion on that trial. During the positioning of the cursor, only the cursor was visible on the display monitor. The monkey was allowed up to 700 msec to position the cursor and was required to maintain it in a stable position inside the electronically defined reward window for 600 msec to receive a reward. The size of the reward window was typically ∼40° of angle around the circumference of the stimulus aperture. The average reward rate across experiments was 82% on trials when no microstimulation was applied and 70% on stimulation trials.

Data analysis

Multiunit directional responses were fit with a Gaussian curve. The preferred direction of the multiunit activity was defined as the peak of the Gaussian.

To quantify the overall effect of microstimulation in each experiment, we first calculated the angle from the center of the target ring to each saccade endpoint. That angle was taken to be the monkey's direction estimate on that trial. In the manual response task, the monkey's direction estimate was taken to be the direction from the center of the stimulus aperture to the final cursor position at the end of each trial. We then calculated the center-of-mass of all direction estimates when only the visual stimulus was presented without microstimulation. Absent any behavioral bias, this center-of-mass would lie at the center of the target ring. We then calculated the center-of-mass for all the direction estimates when microstimulation was applied. The vector extending from the nonstimulated to the stimulated center-of-mass was our measure of the stimulation effect. The magnitude of the vector corresponded to the magnitude of the stimulation effect and the direction of the vector to the stimulation induced direction signal.

To test whether the magnitude of the vector, and thus the magnitude of the stimulation effect, was significantly different from zero, we conducted a permutation test. In each permutation, direction estimates were randomly swapped between stimulated and nonstimulated conditions for the same visual direction. Then the centers-of-mass and the vector were recomputed. The data were permuted 2000 times, and the length of the true vector was compared with the amplitudes of the 2000 vectors obtained from the permuted data. The effect of microstimulation was judged to be significant if the magnitude of the true vector was larger than 95% of the permuted vectors (i.e., significant at the_p_ < 0.05 level). The same center-of-mass calculation and permutation test were also applied to the stimulated and nonstimulated conditions of the 0% coherence block, as well as to the data from the manual response task.

RESULTS

Our primary goal in the present study was to resolve the discrepancy between two previous studies from this laboratory concerning the algorithm by which neural activity in MT generates a percept of visual motion. In a perceptual discrimination task, Salzman and Newsome (1994) found evidence favoring a winner-take-all readout of MT activity. In a second approach, Groh et al. (1997) microstimulated MT during the initiation phase of smooth pursuit eye movements and found clear evidence of a vector-averaging readout of MT, a conclusion that was supported by subsequent behavioral studies of pursuit initiation (Lisberger and Ferrera, 1997).

In this study, we trained monkeys on a direction identification task that required a veridical report of the perceived direction of motion. This task incorporated important elements of both previous studies. Like Salzman and Newsome (1994), we microstimulated direction columns in MT while monkeys provided perceptual reports of the direction of a moving stimulus. Like Groh et al. (1997), however, we presented relatively strong motion stimuli (25–100% coherence) and did not impose perceptual categories on the monkeys' direction estimates, thus removing a level of abstraction between the pattern of neural activity and the perceptual response.

Figure 2B shows saccade endpoints distributed around the target ring (data not shown) for one block of trials from an example experiment. In this block, 18 equally spaced directions of motion were randomly interleaved. Each direction was presented 20 times, and the stimulus was 80% coherent (a vivid suprathreshold motion stimulus). No microstimulation was applied on the trials shown, although it was applied on other trials randomly interleaved within this block. Both the target ring and the motion stimulus were 9° in diameter, centered 10° to the left of the fixation point. We converted these two-dimensional saccade endpoints into one-dimensional direction estimates simply by calculating the direction from the center of the target ring to the endpoint of each saccade. By convention, we assigned saccades to the right side of the target ring a direction of 0°, saccades to the top of the ring a direction of 90°, and so on. Figure 2C plots the mean and circular SD (Zar, 1974) of the monkey's direction estimates as a function of the direction of motion in the visual stimulus for the data shown in Figure2B. The unity slope diagonal represents perfect performance on the task. We quantified the accuracy of the monkey's direction estimates by measuring the absolute value of angular “error” between each visual direction and the corresponding mean direction estimate. Pooling across all 18 visual conditions in this block, the average error was 10.0 ± 7.6° (mean ± SD), and pooling across all blocks for both monkeys, the average error was 17.0 ± 12.6° (mean ± SD). We quantified the variability of the direction estimates by calculating the SD of individual estimates for each visual direction. Pooled across the 18 visual conditions in this block, the average SD was 16.7 ± 4.2° (mean ± SD). Pooling over all blocks, the average SD of the two monkeys' direction estimates was 22.8 ± 11.3° (mean ± SD).

The variability in the monkeys' direction estimates is probably dominated by variability in the saccadic system rather than imprecision at the perceptual level. Previous behavioral experiments conducted under near-optimal conditions (discrete visual targets, exclusion of blatantly errant saccades, elimination of variability caused by initial fixation error) have shown that saccadic endpoints for human subjects exhibit SDs approaching 1° of visual angle at eccentricities similar to those used in our study (van Opstal and van Gisbergen, 1989). Translated into direction around the target ring, this saccade variability alone would lead one to expect a SD in a monkey's direction estimates of ∼11°. Although we observed approximately twice this variability, we did not exclude occasional errant saccades and did not correct for variability in initial fixation position. Furthermore, because no discrete spatial target was presented, the saccades in our task may be more akin to saccades to remembered target locations, which are substantially more variable than saccades to discrete visual targets (Gnadt et al., 1991; White et al., 1994). It therefore seems unlikely that perceptual variability limited performance in most blocks.

Effect of microstimulation

Figure 3 illustrates the effect of microstimulation on the monkey's direction estimates for the same block of trials shown in Figure 2. Figure 3A depicts the direction tuning curve obtained from multiunit responses at this site along with the preferred direction (arrow) computed from the fitted Gaussian (see Methods and Materials). While the monkey performed the direction identification task, we attempted to influence his direction estimates by electrically stimulating (200 Hz, 10 μA) direction-selective neurons in MT. Microstimulation occurred on a randomly selected 50% of trials and precisely coincided in time with the presentation of the visual stimulus.

Fig. 3.

Fig. 3.

The effect of microstimulation on direction estimates. A, Polar plot of multiunit responses at one MT site to eight different directions of motion. The polar angle represents the direction of visual motion; the radius represents discharge rate. The dashed circle indicates the spontaneous discharge rate in the absence of a motion stimulus. The_arrow_ indicates the preferred direction of multiunit activity based on a Gaussian fit to the data. B, Effect of microstimulation on the monkey's direction estimates. Each_black point_ represents the mean direction estimate for one visual direction of motion at 80% coherence (same data as in Fig.2C). Points are staggered radially for visualization purposes. Magenta arrows indicate the shift in the mean direction estimate on microstimulation trials. The black radial arrow indicates the preferred direction for this stimulation site (same as in A). Centers-of-mass for nonstimulated direction estimates (black point) and stimulated direction estimates (magenta point) are displayed relative to the center of the ring. The _magenta arrow_linking the two center-of-mass points indicates the magnitude and direction of the microstimulation effect. C, Effect of microstimulation in 0% coherence block. _Black points_around circumference represent individual direction estimates on nonstimulated, 0% coherence trials. Cyan points_represent direction estimates on stimulated, 0% coherence trials. The_black point inside the ring represents the center-of-mass for nonstimulated direction estimates; the cyan point inside the ring represents the center-of-mass for stimulated direction estimates. The _cyan vector_between the two indicates the magnitude and direction of the microstimulation effect. For an explanation of the asterisk, see Results.

Figure 3B displays the effect of microstimulation at this site. Each black point around the circumference of the circle represents the monkey's mean direction estimate for one direction of the 80% coherent motion stimulus in the absence of microstimulation (same data plotted in Fig. 2C). Each corresponding magenta arrow represents the direction and magnitude of the shift in the animal's mean direction estimate caused by microstimulation. (The points are staggered radially for visualization purposes.) The black radial arrow extending from the center of the ring indicates the preferred direction of the multiunit responses. Clearly, electrical stimulation biased the animal's direction estimates toward the preferred direction of neurons at the stimulation site, and the effect occurred over nearly the entire range of stimulus directions. Consider, for example, the point indicated by an asterisk in the upper right quadrant. The visual direction in this case was 80°. The monkey's mean direction estimate on nonstimulated trials was ∼67°, revealing a modest clockwise bias. On trials when we applied microstimulation, however, the monkey's mean direction estimate was shifted to 43°, a 24° clockwise shift halfway between the mean direction estimate on visual trials and the preferred direction of the stimulation site. Thus, microstimulation produced a substantial effect on the monkey's mean direction estimate for this visual direction.

To quantify the overall impact of microstimulation on the animals' behavior in each block, we computed both the center-of-mass of all the individual direction estimates in the absence of microstimulation (Fig.3B, black point near the center of the ring), as well as the center-of-mass of all direction estimates in the presence of microstimulation (magenta point inside the ring). The magnitude and direction of the vector from the nonstimulated to the stimulated center-of-mass (magenta arrow) provided a measure of the magnitude and direction of the microstimulation effect. Based on the magnitude of this vector, microstimulation significantly shifted the center-of-mass (permutation test, p < 0.05) at 16 of 23 stimulation sites (70%) in monkey R and 14 of 29 sites (48%) in monkey G. Moreover, for the 30 direction identification blocks showing significant microstimulation effects, the direction of the shift in center-of-mass was well correlated with the preferred direction of the multiunit responses at the stimulation site (Fig.4A, black data points) (circular correlation coefficient, 0.8; p < 0.05). (The four open symbols will be discussed in a subsequent section of Results and are not included in the cited correlation.) Across the entire data set, therefore, electrical stimulation biased the animal's direction estimates toward the preferred direction of the stimulation site. Microstimulation also moderately increased the variability of the monkey's direction estimates in this block of trials, as reflected in the SD of the direction estimates across 18 visual conditions (28.7 ± 16.9°; mean ± SD vs 16.7 ± 4.2° on nonstimulated trials). This difference in variability was also evident when we pooled across all blocks with significant stimulation effects (32.5 ± 16.9°; mean ± SD vs 22.8 ± 11.3°).

Fig. 4.

Fig. 4.

A, Direction of the microstimulation effect in the direction identification block as a function of multiunit preferred direction for 30 experiments with significant microstimulation effects (black points). The_open circles_ represent data from four experiments in the manual response task discussed in Results. The _diagonal_is the unity-slope line. The circular correlation coefficient,r, of the two data sets is 0.8. Data points represented by open symbols were not included in calculating the correlation coefficient. B, Direction of microstimulation effect in the direction identification block as a function of direction of microstimulation effect in the 0% coherence block in 19 experiments with significant effects in both blocks. The circular correlation coefficient, r, is 0.94.

At the end of each direction identification block, we tried to obtain a “pure” measure of the direction signal introduced by microstimulation in the absence of a coherent visual direction signal. The monkey performed a second block of trials (“0% coherence block”) in which the visual stimulus on one third of the trials was 0% coherent, and therefore carried no net directional signal. On half of those 0% coherence trials, randomly selected, we applied microstimulation during the visual stimulus presentation. Figure3C shows the impact of microstimulation on the monkey's direction estimates on these 0% coherence trials for the same stimulation site that yielded the data in Figure 3, A and_B_. In the absence of microstimulation (black points), the monkey's estimates were scattered around the target ring with a clear bias toward the side of the ring away from the fixation point. This behavioral bias is reflected in the location of the center-of-mass for these nonstimulated trials (black point inside the ring). Behavioral biases of this magnitude were not unusual for 0% coherence trials that lacked a net visual direction signal. On microstimulation trials (cyan points), however, the animal's direction estimates were clustered on the opposite side of the target ring, with a mean direction of 10°. The cyan point inside the ring indicates the location of the center-of-mass for these trials. The vector from the first to the second center-of-mass again provided an estimate of the magnitude and direction of the stimulation effect within the block. The effect of microstimulation on 0% coherence trials, based on the magnitude of this vector, was significant in 14 of 23 blocks (61%) in monkey R and 6 of 23 blocks (26%) in monkey G (permutation test, p < 0.05). The direction of the vector from the 0% coherence block was highly correlated with the direction of the vector for the main direction identification block (Fig. 4B) (circular correlation coefficient, 0.94;p < 0.05) for 19 experiments in which both blocks showed significant microstimulation effects. This very high correlation level implies that the direction signal introduced by microstimulation remained quite stable across the two blocks of trials. The direction of the vector for the 0% coherence block also agrees closely with the preferred direction of the stimulation site in this example. Indeed, these two independent estimates of the direction encoded by neurons at the stimulation site were well correlated for 20 sites that showed significant microstimulation effects in the 0% coherence block (circular correlation coefficient, 0.87; p < 0.05; data not shown).

Testing the winner-take-all and vector-averaging hypotheses

Are the effects of microstimulation consistent with either the vector averaging or the winner-take-all hypotheses? Evidence for vector averaging could emerge if, on trials when electrical stimulation was applied, the monkey's direction estimates fell between the microstimulated direction and the direction of the visual stimulus, as schematized in Figure 5A(left panel). On the other hand, if the visual stimulus and microstimulation elicit two distinct peaks of activity in MT, the winner-take-all hypothesis predicts that the monkey's direction estimates on any given trial will be determined either by the preferred direction of the microstimulated neurons or by the direction of the visual stimulus (Fig. 5A, right panel). In other words, rather than cooperating to produce intermediate direction estimates, the two signals would compete for perceptual dominance. (Clearly, the net activity in different columns must be somewhat noisy for the winner-take-all mechanism to generate different behavioral responses on different trials. Several sources of noise are possible; in the absence of data we are agnostic about the actual situation in the brain. As long as noise effects are not biased toward any particular subset of possible directions, the predictions of the models are not affected one way or the other.) To evaluate these predictions, it is not sufficient to examine the monkey's mean direction estimates, as in Figure 3B. Instead, we must examine the underlying distributions of individual estimates for each visual direction.

Fig. 5.

Fig. 5.

Illustration of response distributions and hypothesis test. A, Predicted distributions of direction estimates under the vector-averaging hypothesis (left panel) and the winner-take-all hypothesis (right panel). The gray panel represents the display monitor; the black circle represents the target ring; magenta points represent individual saccade endpoints; converging dashed lines represent the monkey's direction of gaze. The _black downward arrow_indicates the direction of visual motion, and the cyan rightward arrow indicates the microstimulated direction.B–E, Distributions of individual direction estimates for four of the visual directions (0, 80, 100, 180°) from Figure 3(ra051b). R values are given where relevant.Black points represent direction estimates for the visual stimulus without microstimulation (visual stimulus coherence was 80%); magenta points represent direction estimates for trials when microstimulation was applied in combination with high coherence visual stimulus; cyan points represent direction estimates on trials when microstimulation was applied during a 0% coherent stimulus. Dashed radius represents the direction of visual motion. Solid black and cyan radii_represent the median direction estimates for the visual and 0% coherence trials, respectively. The median direction estimate for 0% coherence trials was 17° in this experiment, and the median direction estimates for visual trials, from top to bottom, were 2, 62, 85, and 193°. F–I, Distributions of direction estimates for four visual directions from an additional experiment in another monkey (gr026j). R values are given where relevant. Median direction estimate for 0% coherence trials was 263° in this experiment, and the median direction estimates on visual trials, from_top to bottom, were 248, 152, 149, and 75°. Visual stimulus coherence was 100%.

Figure 5B–E displays the distributions of individual direction estimates for four of the 18 visual directions from Figure3B. Black points represent the monkey's direction estimates in response to the high coherence motion stimulus alone (“visual trials”), magenta points represent direction estimates when microstimulation was applied in combination with a high coherence motion stimulus (“interaction trials”), and cyan points represent direction estimates when microstimulation was applied during a 0% coherent stimulus (“0% coherence trials”) as plotted in Figure3C. In this display, the black points provide a pure estimate of the influence of the visual direction signal, the cyan points provide a pure estimate of the influence of electrical stimulation in the absence of a visual direction signal, and the magenta points reveal the interaction of visual and electrical signals.

For the trials displayed in Figure 5B, the direction of the visual stimulus and the preferred direction of neurons at this site were nearly aligned. Predictably, microstimulation had little impact on the monkey's direction estimates in this case. For the trials displayed in Figure 5C, however, the direction of the visual stimulus was 80°, 73° away from the preferred direction of neurons at this site. Visual inspection reveals that the magenta points (interaction trials) cluster between the distributions of black and cyan points, as schematized in the left panel of Figure 5A. The data in Figure 5D follow a similar pattern, when the visual direction was 100°, 93° away from the preferred direction of neurons at this site. Finally, for the trials displayed in Figure5E, the direction of the visual stimulus was 180°, 173° away from the preferred direction. In this case, the distribution of magenta points appears to be bimodal, with one mode corresponding to the visual direction and one to the microstimulated direction, as schematized in the right panel of Figure 5A. Figure5F–I displays a similar pattern of results for an experiment in a second monkey.

We quantified the extent to which the distribution of direction estimates on interaction trials lay intermediate between the distribution for visual trials and the distribution for 0% coherence trials. We will describe our method in the context of Figure5C. We first found the median direction estimate for the visual trials (black radius) and for the 0% coherence trials (cyan radius). These two medians, 75° and 14°, respectively, in this example, form a wedge indicated by the arc in the figure (i.e., the shortest angle between the medians). Half of all the visual trials and half of all the 0% coherence trials lie between the medians in this wedge. Assuming that the visual stimulus and microstimulation elicit distinct peaks of neural activity in MT (see Discussion), a pure winner-take-all hypothesis predicts that individual direction estimates on interaction trials are drawn either from the same underlying distribution as the visual trials (when the visual direction determines the direction percept) or from the same underlying distribution as the 0% coherence trials (when the microstimulated direction determines the direction percept). In that case, we would expect the proportion of direction estimates that lie within the wedge on interaction trials to be the same as the proportion on visual and 0% coherence trials. We calculated a ratio, R, in which the denominator is the expected proportion of trials lying in the wedge under the winner-take-all hypothesis: the proportion of all visual and 0% coherence trials in the wedge. In the present example, the denominator is 0.5, but can be <0.5 when the angle between the medians is small relative to the variability in the direction estimates. The numerator of the ratio is the actual proportion of direction estimates on interaction trials that lie in the wedge. The ratio will tend toward unity according to a pure winner-take-all hypothesis. Vector averaging, on the other hand, predicts that the ratio will be greater than unity, because the direction estimates on microstimulation trials will tend to cluster between the two medians. (We have confirmed the behavior of this statistic through extensive simulations. These predictions depend on the assumption that the distribution of activity elicited by the visual and electrical activity are, on average, reasonably symmetric about the median; further assumptions underlying this differential prediction will be considered in Discussion).

Obviously, the two hypotheses do not make differential predictions when the visual and microstimulated directions are directly aligned (0° angular difference). Furthermore, when the visual and stimulated directions are directly opposite (180° angular difference), both winner-take-all and vector averaging could predict bimodally distributed direction estimates. How far from 180° angular difference must we proceed to test the models reliably? In essence, this question concerns the accuracy of our estimates of the perceptual effects of the visually and electrically evoked activity in MT (i.e., the black and cyan radii in Fig. 5C). If these estimates were extremely precise, the winner-take-all and vector averaging models would make differential predictions even for an angular difference of 179°. In general, however, our estimates of the behavioral effects (as represented by the radii) are accurate to no more than 5–10°. Across the 18 visual stimulus conditions in Figure 2C, for example, the average SE of the monkey's direction estimates was 3.74°, but the behavior in this experiment was less variable than average. More often the SEs fell between 5 and 10°. To be conservative, therefore, we restricted the analysis to stimulus conditions for which the angular difference between the black and cyan radii was ≥20° and ≤160°.

For all 19 experiments with significant microstimulation effects in both the main block and the 0% coherence block, we calculated the value of R for those visual conditions that showed statistically significant shifts in the animal's direction estimates. Figure6 plots the mean and SE of _R_as a function of the angular separation between the visual direction and the microstimulated direction (as derived from the 0% coherence block). R values are averaged within 20° bins and plotted at the angular separation corresponding to the center of each bin. Asterisks indicate mean values that are significantly greater than unity (t test; p < 0.05). When the visual direction and the microstimulated direction differed by >20 and <140°, the monkeys' direction estimates on microstimulation trials tended to lie intermediate between the visual and microstimulated directions, broadly consistent with a distributed coding scheme like vector averaging (winner-take-all can be conclusively rejected only for angular separations that yield distinct peaks of visually and electrically induced activity in MT; see Discussion). When the visual and microstimulated directions differ by 140–160°, however, the ratio is near unity, consistent with the winner-take-all prediction.

Fig. 6.

Fig. 6.

Quantitative test of hypotheses for 19 experiments. The abscissa is the angular difference between the electrically evoked direction (based on 0% coherence trials) and visual direction. The ordinate is the proportion of direction estimates on interaction trials (magenta points in_B_) that lie between the two median angles shown in_B_, divided by the proportion predicted by the winner-take-all hypothesis. R values were averaged within 20° bins and plotted at the angle corresponding to the center of each bin. The plot shows mean and SE of R for differences in visual and microstimulated direction of 20–160°. The_dashed horizontal line_ indicates the value of_R_ expected for a pure winner-take-all model.Asterisks indicate mean values that are significantly greater than unity (t test; p < 0.05).

Stimulus coherence

In previous microstimulation studies using the variable-coherence random dot stimuli, we rarely observed microstimulation effects for motion stimuli of >50% coherence (Salzman et al., 1992; Murasugi et al., 1993a,b; Salzman and Newsome, 1994; Celebrini and Newsome, 1995). We speculated that microstimulation was unable to influence direction judgments for substantially suprathreshold motion stimuli because directional signals carried by neurons outside of MT (or perhaps by nonstimulated columns within MT) overwhelmed the electrical signal created by microstimulation. In the current experiments, however, our motion identification task permitted us to detect more subtle microstimulation effects than was possible with the relatively crude categorization tasks used previously. Our monkeys were free to report even slight changes in the perceived direction of motion, and we also accumulated larger data sets for each motion condition than was the case in our previous studies, increasing our statistical power. Across our data set, we obtained several significant microstimulation effects at very high coherences, including four experiments at 100% coherence and two experiments at 80% coherence (Figs. 3, 5B–E). Thus, microstimulation of single sites in MT can influence perceptual judgments even for the most powerful motion stimuli. This does not, of course, exclude the possibility that other visual areas or columns in MT may also contribute to the motion judgments.

Manual response task

The present results, in conjunction with the results of our previous studies (Salzman et al., 1992; Salzman and Newsome, 1994), strongly suggest that neural activity in MT helps shape the perception of visual motion. In this and all our previous studies, however, the monkeys have always used eye movements to report their perceptual judgments. If the sensory information encoded by MT truly supports motion perception, then that sensory information should be globally available to other motor systems, not just to the oculomotor circuitry. To explore this possibility, we trained a third monkey on a variant of the direction identification task (Fig.7A) in which he reported his direction estimates with a hand movement rather than a saccade (see description in Materials and Methods).

Fig. 7.

Fig. 7.

The effect of microstimulation on direction estimates in a manual response task. A, Schematized illustration of the task. A monkey is seen from behind viewing a display monitor. He directs his gaze (converging dashed lines) at the fixation point (cross) while a random dot motion stimulus is presented inside the receptive field (dashed circle) of the neurons under study. After the fixation point and the motion stimulus are extinguished, the monkey moves a manipulandum to position the response cursor (black disk) in the direction of motion of the stimulus. B, Mean and circular SD of the monkey's direction estimates as a function of the direction of the visual stimulus in the absence of microstimulation.C, Multiunit direction tuning curve for this stimulation site. The arrow indicates the preferred direction based on a Gaussian fit to the data. D, Effect of microstimulation on manually reported direction estimates. Each_black point_ around the circumference of the circle represents the mean direction estimate for one of eight possible directions of motion on nonstimulated trials in this experiment.Light gray arrows indicate the direction and amplitude of the effect of microstimulation on those mean direction estimates. The black point and the gray point in the_middle_ of the circle indicate the center-of-mass of the directional judgments on nonstimulated and stimulated trials, respectively (as in Fig. 3). The _radial black arrow_indicates the multiunit preferred direction.

Figure 7B displays the monkey's mean direction estimates as a function of visual stimulus direction for one set of nonstimulated trials in the manual response task. Plainly, the monkey's estimates, on average, were well matched to the direction of stimulus motion, just as with the saccadic version of the task.

Figure 7C shows the multiunit direction tuning for one stimulation site in MT. The arrow indicates the preferred direction (252°) for the site based on a Gaussian fit to the data. Figure7D illustrates the effect of microstimulation on the monkey's direction estimates for one block of trials in which the motion stimulus was 25% coherent. Black points around the circumference indicate the animal's mean direction estimates on visual trials for each of eight directions of motion spanning 360° at 45° intervals. The corresponding light gray arrows indicate the direction and amplitude of the shift in the mean direction estimates caused by microstimulation. The centers-of-mass for stimulated and nonstimulated direction estimates are shown within the circle. As in the saccadic version of the task, microstimulation clearly biases the monkey's direction estimates toward the preferred direction for the stimulation site (radial arrow). Indeed, microstimulation exerted a significant effect on the monkey's direction estimates for four of nine stimulation sites (p < 0.05; permutation test), and the direction of the microstimulation effect was in good agreement with the preferred direction of neurons at the stimulation site (Fig.4A, open circles). Thus, electrical stimulation of MT can influence monkeys' perceptual judgments of motion direction across different effector systems, indicating that the sensory information encoded by MT activity is globally available to motor control systems.

DISCUSSION

We examined the effects of microstimulation in MT on performance in a motion identification task. In our primary analysis, we tested the predictions of two competing algorithms, vector averaging and winner-take-all, for reconstructing the direction of motion of a visual stimulus from the profile of neuronal activity in MT. We found that, in the context of a perceptual task, neurons with preferred directions as disparate as 140° can cooperate to influence a monkey's behavioral estimates of direction. Neurons with more disparate preferred directions, however, seem to compete in determining direction estimates.

Before definitively interpreting our findings in terms of the two hypotheses, however, we must consider the profile of activation that results from combining electrical stimulation with visual activation of MT neurons. Assuming electrically evoked and visually evoked activity are summed, then the combination of the two could result in a single peak of neural activity occurring in neurons whose preferred direction is intermediate between the visual and microstimulated directions. Under these circumstances, winner-take-all and vector averaging would predict similar outcomes (i.e., direction estimates lying intermediate between the visual and stimulated directions). Clearly, the two hypotheses only make differential predictions if the visual stimulus and microstimulation evoke distinct peaks of activity in MT, as schematized in Figure 1B. We did not measure the profile of activation directly in this study, but we draw insight about the distribution of activity from previous physiological and microstimulation studies.

Treue et al. (2000) showed that the responses of MT neurons to two superimposed random dot motion stimuli at 100% coherence could be predicted from the sum of the their responses to the two stimuli separately. Because of the breadth of direction tuning in MT (∼90° at half-height), directional differences <90° evoked a single peak of activity in MT. Because Britten and Newsome (1998) showed that the breadth of direction tuning does not depend significantly on stimulus coherence, the conclusions reached by Treue et al. (2000) should apply to the range of coherences used in the present study. Therefore, if microstimulation were to activate as much of the functional architecture for direction as a visual stimulus does, vector averaging and winner-take-all would only make differential predictions when the visual and microstimulated directions differed by ≥90°. We believe, however, that the electrical stimulation used in our studies activated a smaller span of direction columns than would a visual stimulus. From direct measurements made by Stoney et al. (1968) in monkey motor cortex, we know that 10 μA current pulses (the value used in the bulk of our experiments) activate neurons within ∼85 μm of the electrode tip. Because we selected microstimulation sites in which the preferred direction was approximately constant for 200 μm of electrode travel, neurons activated by our stimulation currents should be substantially restricted to single direction columns.

Additional inferences can be drawn from previous microstimulation studies. Most importantly, Salzman and Newsome (1994) found evidence favoring the winner-take-all hypothesis for angular differences of ≥90° using the same microstimulation parameters and site-selection methods used here. (They did not test smaller angular differences between the microstimulated and visual directions.) Conservatively, then, our stimulation parameters should elicit two distinct peaks of activity in MT for 90° differences between the visual and stimulated directions, and perhaps for smaller angular differences as well.

Even when visual and electrical stimulation indeed create two peaks of activity in MT, interactions between the two sources of neural activity could still complicate interpretation of our data. Simple summation of visually and electrically induced activity, for example, could shift the two peaks toward each other in the direction map, leading to values of the computed ratio, R, greater than unity even in a winner-take-all scenario. Again, if electrical stimulation activates as much of the function architecture for direction in area MT as does a visual stimulus, such a shift could be substantial at angular separations of 90°, complicating the interpretation of our results. At angular separations of 140°, however, such a shift would be negligible assuming typical direction tuning curves with a width at half-height of 90°.

Very conservatively, then, we can draw our strongest conclusions when the visual and microstimulated directions differ by 120–140°. At this angular separation, we see clear evidence that the visual and microstimulated directions cooperate to determine the monkeys' direction estimates in this range, inconsistent with a winner-take-all model. If, as we suspect, electrically elicited activity is in fact more restricted than visually elicited activity (see above), we are able to draw conclusions for a larger range of angular separations: we see clear evidence of interaction between the two direction signals from 80°-140° of angular difference. These findings are inconsistent with the predictions of the winner-take-all hypothesis and favors a distributed representation like vector averaging. When the visual and microstimulated directions are more disparate (140–160°), however, a winner-take-all algorithm seems to operate.

We emphasize that vector averaging is just one of several possible distributed codes, all of which differ from the winner-take-all algorithm in that neurons with disparate preferred directions cooperate to influence perceptual judgments. To take just three examples, vector summation (Georgopoulos et al., 1986; Groh et al., 1997), intersection of constraints (Adelson and Movshon, 1982), and template fitting (Deneve et al., 1999) all involve combining disparate direction signals to form a representation of stimulus direction. Although we can conclude from our data that neurons with preferred directions differing by as much as 140° can cooperatively influence direction estimates, extracellular microstimulation in behaving monkeys may be too crude a method to distinguish among different algorithms for pooling such signals.

Strictly speaking, of course, our results pertain only to the restricted set of conditions we have actually tested: the interaction of a single visual stimulus and a single artificially elicited directional signal. Visual–visual interactions can result in very different outcomes from those we have demonstrated here. In motion transparency, for example, superimposed motion stimuli retain distinctive identities and are not resolved into a single direction of motion. For “plaid” stimuli composed of superimposed sinusoidal gratings, two motion vectors can indeed be resolved into a single direction of motion, but the resolution follows completely different rules from those considered here (Adelson and Movshon, 1982). Our experiments were directed specifically toward probing the “read-out” mechanisms operative when a single visual pattern is present. The power of our approach is that the exploratory perturbation is introduced to the system via a microelectrode, which permits relatively precise localization of the perturbation within the cortex. In contrast, perturbations introduced through a visual display, as in motion transparency, elicit changes in neural activity throughout vast swathes of the visual cortex. Our results, therefore, are more likely to reveal mechanisms by which MT activity itself is read out.

Previous microstimulation results

Twice previously, we have examined these hypotheses by applying microstimulation in MT. In the psychophysical task of Salzman and Newsome (1994), microstimulation influenced monkeys' behavioral reports in a manner consistent with winner-take-all, at least when the visual and electrical vectors differed by ≥90°. In a smooth pursuit task (Groh et al., 1997), however, microstimulation produced clear vector-averaging effects, even for large angular separations of the two vectors. In the present study, we have combined elements of both previous tasks in an effort to resolve the apparent discrepancy in the results. On one hand, our monkeys performed a psychophysical judgment about the direction of a motion stimulus after an extended viewing interval as in our previous psychophysical studies. On the other hand, our procedures in the current experiments resemble the earlier pursuit study by using strong motion stimuli and permitting veridical judgments of the motion direction.

Unlike Salzman and Newsome (1994), we find clear evidence of a distributed code that encompasses neurons with preferred directions as disparate as 140°. What are the key differences between the two psychophysical tasks? Before the current study, it was plausible that different algorithms prevailed in different behavioral contexts: the winner-take-all algorithm in the context of perceptual tasks and vector averaging in the context of pursuit initiation. We can now eliminate that possibility. Another possibility was that the difference lay in temporal integration of the motion signals: with an extended integration interval available, as in the psychophysical tasks, monkeys could segregate the visual and microstimulated motion signals rather than combining them in a vector average. Our data also eliminate that possibility for the range of directions over which we found evidence of a distributed representation.

Another key difference between the present task and the task used bySalzman and Newsome (1994) is that the present task did not force the monkeys to bin their judgments into experimenter-imposed direction categories. Because the categories used by Salzman and Newsome (1994)were broad (45°), the monkeys may have been unable to report modest shifts in perceived direction caused by microstimulation. Only direction estimates that differed from the visual direction by >22.5° would have been cast in a different bin than the visual stimulus itself. Under these circumstances, many of the effects observed in the present study would have been difficult to detect (Fig.3B). Because winner-take-all effects necessarily involve casting direction estimates into different direction categories, the experimental approach used by Salzman and Newsome (1994) may have preferentially selected for winner-take-all effects and tended to overlook vector averaging effects. In contrast, the task in the present study allowed monkeys to report more directly and accurately the perceptual effects of microstimulation, making it possible to detect more subtle vector-averaging effects.

Unlike Groh et al. (1997), we found clear evidence of winner-take-all behavior when the visual and microstimulated directions differed by ≥140°. Both studies used strong motion stimuli and did not force monkeys to categorize their behavioral responses. One important difference between the studies, however, is the amount of time the animals were allowed to integrate motion signals before reacting. In the present study, they were given a 1 sec viewing interval, whereas in the pursuit task motion computations occurred in <200 msec preceding the initiation phase of pursuit. While this difference in viewing interval does not seem to have made a difference when the visual and stimulated directions differed by <140, it could be that longer viewing times allow one or the other direction signal to dominate when the visual and stimulated directions were nearly opposite. Indeed, behavioral studies using two competing visual pursuit targets reveal vector averaging during pursuit initiation, but winner-take-all during the maintenance phase of pursuit (Lisberger and Ferrera, 1997). It would be intriguing to test whether very brief viewing intervals in the perceptual task would lead to vector averaging over an even wider range of directions, as was observed in the pursuit task.

Parker and Newsome (1998) recently advanced seven empirical criteria that would support a hypothesized link between the activity of specific neurons in the CNS and performance on a particular perceptual task. Until the present study, no hypothesized link satisfied all seven criteria. The link between MT neurons and directional judgments, for example, had not been shown to be independent of the particular motor response used by the observer to report perceptual judgments. We have now shown that microstimulation of MT can influence direction estimates whether the operant response is an eye movement or a hand movement, and MT may therefore be considered to satisfy all seven criteria. To our knowledge, MT is the first brain area to be this tightly linked to perception.

Footnotes