Neural Mechanisms of Visual Working Memory in Prefrontal Cortex of the Macaque (original) (raw)
Articles
Journal of Neuroscience 15 August 1996, 16 (16) 5154-5167; https://doi.org/10.1523/JNEUROSCI.16-16-05154.1996
Abstract
Prefrontal (PF) cells were studied in monkeys performing a delayed matching to sample task, which requires working memory. The stimuli were complex visual patterns and to solve the task, the monkeys had to discriminate among the stimuli, maintain a memory of the sample stimulus during the delay periods, and evaluate whether a test stimulus matched the sample presented earlier in the trial. PF cells have properties consistent with a role in all three of these operations. Approximately 25% of the cells responded selectively to different visual stimuli. Half of the cells showed heightened activity during the delay after the sample and, for many of these cells, the magnitude of delay activity was selective for different samples. Finally, more than half of the cells responded differently to the test stimuli depending on whether they matched the sample. Because inferior temporal (IT) cortex also is important for working memory, we compared PF cells with IT cells studied in the same task. Compared with IT cortex, PF responses were less often stimulus-selective but conveyed more information about whether a given test stimulus was a match to the sample. Furthermore, sample-selective delay activity in PF cortex was maintained throughout the trial even when other test stimuli intervened during the delay, whereas delay activity in IT cortex was disrupted by intervening stimuli. The results suggest that PF cortex plays a primary role in working memory tasks and may be a source of feedback inputs to IT cortex, biasing activity in favor of behaviorally relevant stimuli.
The ability to actively hold an item in memory for a short time is a defining feature of “working memory.” In monkeys, visual working memory has been studied in delay tasks, such as delayed matching to sample (DMS), which require that a memory be held during a brief delay period. At least two lines of evidence indicate that prefrontal (PF) cortex plays an important role in working memory. First, lesions or reversible deactivation of lateral PF cortex in monkeys impair performance on delay tasks (Mishkin, 1957; Gross and Weiskrantz, 1962; Mishkin et al., 1969; Goldman and Rosvold, 1970;Goldman et al., 1971; Passingham, 1975; Mishkin and Manning, 1978). Second, many PF cells are activated by specific stimuli during the delay interval of such tasks (Fuster and Alexander, 1971; Kubota and Niki, 1971; Fuster, 1973, 1985; Niki, 1974a,b,c; Niki and Watanabe, 1976; Fuster et al., 1982; Kojima and Goldman-Rakic, 1982; Quintana et al., 1988; Funahashi et al., 1989; di Pellegrino and Wise, 1991;Quintana and Fuster, 1992; Wilson et al., 1993).
Another region implicated in working memory, at least for visual patterns, is inferior temporal (IT) cortex. Lesions or cooling of IT cortex impair performance on DMS tasks (Horel et al., 1987; Gaffan and Murray, 1992). We previously studied working memory in IT cortex using a modified DMS task, in which the sample stimulus was followed by a sequence of several test stimuli and the animal was rewarded for indicating when one of the test stimuli matched the sample (Miller et al., 1991b, 1993; Miller and Desimone, 1994). Consistent with other studies (Gross et al., 1979; Mikami and Kubota, 1980; Baylis and Rolls, 1987; Riches et al., 1991; Eskandar et al., 1992; Vogels et al., 1995), we found that the memory of the sample was reflected in the responses to the subsequent test stimuli. The responses of some cells were suppressed by any stimulus repetition, behaviorally relevant or not, whereas other cells gave enhanced responses only to the test stimulus that matched the sample. The latter cells might mediate the animal’s decision about whether a current stimulus matched an item in working memory. However, although the memory of the sample clearly influenced IT responses to subsequent stimulus presentations, we failed to find an explicit neuronal representation of the sample that was maintained throughout the trial. Some cells did show sample-selective activity in the delay after the sample; however, this activity was abolished when the first test stimulus appeared in the sequence.
Our failure to find persistent sample-selective delay activity in IT cortex led us to search for cells with such properties in another area. PF cortex was an obvious possibility, but it was unknown whether PF delay activity, like IT delay activity, is disrupted by intervening stimuli. To test the effects of intervening stimuli, as well as to compare PF and IT responses to the test stimuli themselves, we recorded from PF cells using the same DMS tasks that we used in IT cortex.
MATERIALS AND METHODS
Subjects and surgical procedures. Two rhesus monkeys weighing 7–9 kg were used. The general methods were reported previously (Miller et al., 1993) and will only be briefly described here. Before surgery, the monkeys were placed in a plastic stereotaxic machine and scanned with magnetic resonance imaging (MRI). The MRI images were used to determine the stereotaxic coordinates of the arcuate and principal sulci. Under aseptic conditions, a head post, recording chamber, and scleral eye coil for monitoring eye position (Robinson, 1963) were implanted while the monkeys were under general anesthesia. The recording chamber was implanted tangential to the cortical surface, centered above the inferior convexity of the PF cortex. The animals received antibiotics and analgesics after surgery.
Recording techniques. Neural activity was recorded using tungsten microelectrodes. Waveforms from individual cells were isolated using either an on-line spike-sorting system (Signal Processing Systems, Prospect) or an off-line spike sorting system (Datawave Technologies). While the monkey performed the task, the electrode was advanced until the activity of at least one neuron was isolated. If the neuron exhibited any change in activity at any time during the trial (assessed by audio monitor and on-line histograms), data collection was initiated. Otherwise, the electrode was advanced to the next neuron.
Behavioral task. We used the same two versions of the DMS task that we used previously in IT cortex (Miller et al., 1991b, 1993;Miller and Desimone, 1994), which are illustrated diagrammatically in Figure 1. For both versions, the trial started with the monkey grasping a metal bar and fixating a small spot of light (the fixation target) at the center of a computer screen. The monkey was required to maintain fixation on the fixation target for 300 msec before presentation of the first stimulus and to maintain fixation throughout the trial.
Fig. 1.
Outline of the DMS task. An example of a standard trial is illustrated in the top row, and an example of an_ABBA_ trial is shown in the bottom row. The number of nonmatching test items between the sample and the matching test item was random from trial to trial, ranging from zero to four. Although the stimuli are shown as line drawings, the actual stimuli in the experiment were color digitized pictures.
The first stimulus of each trial was the sample, which was followed by a sequence of one to five test stimuli, terminating with a stimulus that matched the sample. When the matching stimulus appeared, the monkey was required to release the bar within 900 msec of stimulus onset to receive a juice reward. Each stimulus was on for 500 msec, followed by a 1000 msec delay before the onset of the next stimulus. The match stimulus was extinguished as soon as the animal released the bar. The number of test stimuli intervening between the sample and final match ranged from 0 to 4 and was randomly determined on each trial.
One version of DMS we called the standard task. In this task, only the sample–match stimulus appeared twice in the sequence. The nonmatching test stimuli were different from the sample and from each other. For example, sample stimulus “A” might be followed by “B … C … D … A.” The monkey was required to make its behavioral response to the second “A.”
The other version of DMS we called the ABBA task. For_ABBA_ trials, one of the nonmatching stimuli appeared twice in the sequence. For example, a sample stimulus, “A,” might be followed by “B … B … C … A.” The animal was required to ignore the repetition of the nonmatching test stimulus (BB) and respond only to the match (A). The ABBA version consisted of ABBA trials randomly intermingled with_standard_ trials. Thus, ABBA trials are similar to standard trials except that they provide an additional control; they allow us to distinguish the specific neuronal response differences caused by a test stimulus matching the sample from the nonspecific effects of one stimulus being a repetition of another in the sequence.
We recorded from the PF cortex of one monkey performing the_ABBA_ task and a second monkey performing the_standard_ task only. Unfortunately, this latter monkey died before it could be tested on ABBA trials.
Stimuli. The stimuli were a set of more than 500 complex, two-dimensional, multicolored images presented on a computer screen. The images were digitized from magazines, objects in the laboratory, etc. They were the same set of stimuli that we used to study properties of IT neurons (Miller et al., 1991b, 1993; Miller and Desimone, 1994). They subtended 1–3° of visual angle on a side and were presented at the center of gaze. For each daily recording session, six pictures from the set were randomly chosen as stimuli. Thus, the same set of stimuli was occasionally used for more than one cell. The stimuli were not used again until the entire set had been exhausted. Each of the six stimuli appeared as a sample–match on some trials and as a nonmatch on other trials.
Data analysis. PF responses were calculated over a 200 msec time interval beginning 90 msec after stimulus onset. The beginning of the time interval was chosen to correspond with the typical stimulus-evoked response latencies of PF neurons, and the end was chosen to occur before the animal’s behavioral response. For analyses of delay activity, we calculated the firing rate over the last 600 msec of the 1000 msec delay interval. We did not include the first portion of the delay in the analyses, so that any responses related to the offset of the preceding stimulus would be excluded. Spontaneous activity was calculated over a 300 msec time window preceding the fixation of the fixation target that started the trial.
Because each trial ended with a matching stimulus, there was a maximum of three intervening stimuli that could precede a given nonmatch stimulus, but a maximum of four intervening stimuli that could precede a match. Therefore, to equate for the number of intervening stimuli, responses to matching stimuli on trials with four intervening stimuli were excluded from all analyses of match versus nonmatch responses.
Statistical analyses. Visual responses and delay activity were appraised using t tests and ANOVAs, evaluated at_p_ < 0.05. We could not calculate “tuning curves” for the stimuli, because they were highly complex and did not form an orderly set. We therefore used both ANOVA and a discriminant analysis to quantify the stimulus selectivity of the neuronal responses and delay activity. (for details, see Miller et al., 1993). Although the ANOVA and discriminant analysis provided a statistical measure of how well the neuronal responses distinguished among the stimuli, we made no attempt to determine the “critical features,” if any, of the stimuli for which the cells may have been selective.
IT neurons. The monkey performing the ABBA task had participated in a previous study of the properties of IT neurons using the same task (Miller and Desimone, 1994). This afforded the opportunity to directly compare properties of IT and PF neurons in the same monkey, avoiding problems inherent in comparisons across animals, such as subtle differences in training history or the behavioral strategy used by the animal that might affect neuronal properties. For this comparison, we analyzed data from neurons recorded previously in the perirhinal portion of IT cortex in this monkey. These neurons comprised part of the data set used in Miller and Desimone (1994), which reported match–nonmatch response differences in IT cortex.
Localization of recording sites. Recording sites in both monkeys were localized using MRI. In addition, we confirmed the location of the sites in the monkey performing the ABBA task by injecting fluorescent latex beads into representative recording sites and processing the brain histologically.
RESULTS
Anatomical location of penetrations and general properties of PF neurons
The recording sites in PF cortex were located on the inferior convexity, ventral to the principal sulcus and anterior to the inferior arcuate sulcus. Figure 2 shows the location of the recording sites from both monkeys. We recorded from a total of 264 PF neurons, of which 109 appeared to be completely unresponsive during initial testing and were not studied further. The remaining 145 (55%) were studied with the full DMS task and are the subject of this report. Ninety-eight of these cells were recorded from the monkey performing the ABBA task, and 47 were recorded from the monkey performing the standard task.
Fig. 2.
Location of recording sites in both monkeys.amt, Anterior middle temporal sulcus; sts, superior temporal sulcus; ls, lateral sulcus; cs, central sulcus; as, arcuate sulcus; ps, principal sulcus; orb, orbital sulcus. Scale bar, 1 cm. Shaded areas indicate extent of recording sites.
Responses to visual stimuli
We determined whether a cell had a significant visual response by using a paired t test (evaluated at p < 0.05) to compare the cell’s firing rate during presentation of all test stimuli with its firing rate during the delays preceding the test stimuli. Based on this criterion, 76% of the cells (110/145) were visually responsive. The majority (75/110, or 68%) gave excitatory responses, and the remainder (35/110, or 32%) were inhibitory. The incidence of visual responsiveness was similar in the two monkeys (78%, or 76/98, of the cells from the ABBA monkey; 72%, or 34/47 of the cells from the standard monkey). Six of the visually unresponsive cells had responses clearly linked to the motor response (bar release), and all of these cells were located close to the inferior arcuate sulcus.
Many of the neurons were stimulus-selective in that they responded better to some stimuli than to others. To assess this quantitatively, we compared responses to the test stimuli using an ANOVA for each visually responsive cell. According to this test, the responses of 37% (41/110) of the cells varied significantly according to the stimulus. The incidence of stimulus-selective, visually evoked responses was similar in the two monkeys (38%, or 29/76, of the responsive neurons from the ABBA monkey; 35%, or 12/34, of the responsive neurons from the standard monkey).
Having determined how many cells were visually responsive, we next calculated the probability that an arbitrary stimulus would elicit a response from any PF neuron. For each cell, we applied a _t_test to the responses to each test stimulus compared with the activity in the delay preceding the test stimulus. Of the 870 stimuli used as test stimuli (145 neurons × 6 stimuli), more than half elicited a visual response (472/870, or 54%). That is, there was better than a 50% chance that a given stimulus would cause at least a small but significant response from a given PF neuron. Most stimuli elicited excitatory responses (356/472, or 75%).
Fixation-related responses
The responses of some PF neurons were related to the animal fixating the fixation target at the start of the trial. Two examples are shown in Figure 3. The neuron illustrated in the top of the figure gave a phasic response when the animal achieved fixation followed by a phasic response to the sample stimulus. By contrast, the neuron illustrated in the bottom of the figure showed a sustained increase in activity after the animal achieved fixation followed by phasic responses to the sample stimuli that appeared to be added to this higher sustained rate. Fourteen percent (21/145) of the cells had fixation-related responses.
Fig. 3.
Example of fixation-related responses. The_top histogram_ is from a cell with a phasic response at the time that the animal fixated the fixation target (time = 0). The_bottom histogram_ is from a cell with a sustained change in firing rate after fixation. Bin width, 10 msec. The horizontal line indicates time of sample presentation. Time is in milliseconds.
Delay activity
During the delay intervals of the DMS task, the monkeys viewed a blank screen while maintaining a memory of the sample stimulus. During these delays, many PF neurons showed high levels of activation (delay activity). For each cell, we compared the average firing rate across the delay intervals with the spontaneous firing rate before the start of the trial by using a paired t test. More than half of the PF neurons (56%, or 82/145) showed significantly higher activity during the delay intervals compared with the spontaneous firing rate. For these cells, the average baseline firing rate was 11.9 spikes/sec, and the average level of delay activity was 16.0 spikes/sec. On average, delay activity was a 42% increase over baseline firing rate (SE = 3.3%, range 4.9–351.2%).
Sample-selective delay activity
For many cells, the magnitude of delay activity varied depending on which stimulus had been used as a sample at the start of the trial, i.e., the delay activity was sample-selective. Examples of responses from such a cell are shown in Figure 4. We assessed this for each cell in the population by computing a two-way ANOVA on the delay activity in each interval during the trial. One factor was the stimulus that was used as the sample on that trial (SAMPLE factor), and the other factor was the order of the delay interval in the sequence (INTERVAL factor, i.e., the delay interval after the sample stimulus, the delay interval after the first test stimulus, etc.).
Fig. 4.
Response histograms of a PF neuron showing sample-selective delay activity. The gray bars indicate when each of the stimuli was presented. Time 0 indicates onset of the sample. Bin width, 10 msec. The baseline firing rate of this neuron was 13 spikes/sec.
The SAMPLE factor was significant for 28% (40/145) of the neurons, indicating that, for these cells, the overall amount of delay activity across the trial varied for the different samples. Because only six randomly chosen stimuli were tested on each cell, this figure probably represents a lower bound on the true incidence of stimulus-selective delay activity. The incidence of sample-selective delay activity was significantly greater in the monkey performing the ABBA task (34%, or 33/98) than in the monkey performing the _standard_task (15%, or 7/47, χ2, p = 0.012). Approximately half of the total cells in both monkeys had a significant effect of INTERVAL, i.e., differing amounts of delay activity across the different delay intervals (ABBA monkey: 56/98, or 57%; standard monkey: 26/47, or 55%). Only 11% (16/145) of the total cells showed a significant interaction between the two factors, including 8 of the 40 cells with sample-selective delay activity (e.g., see Fig. 4). Thus, although the overall amount of delay activity often varied across the different delay intervals in the trial, the relative amount of delay activity after different samples appeared to be largely preserved across intervals.
Although the ANOVA indicated that many cells had sample-selective delay activity averaged across all delay intervals, an important question was whether this stimulus selectivity was maintained during each delay interval in the trial sequence. We approached this question in several ways. In a previous study of IT neurons (Miller et al., 1993), we computed the activity in each delay interval on trials that began with a cell’s preferred sample stimulus and compared that with the delay activity on trials that began with the cell’s least-preferred sample stimulus. Unfortunately, this approach was not practical in PF cortex, because many PF neurons with delay activity either did not have visually evoked responses to the samples or responded nonselectively. Therefore, for each cell with significant sample-selective delay activity (according to the ANOVA described above), we determined separately which sample resulted in the greatest delay activity averaged across all delay intervals and which sample resulted in the least activity averaged across all intervals. Figure 5shows these responses after the best and worst samples separately, averaged across the population of cells. Although the delay activity is disrupted by stimulus-evoked responses to the test stimuli, the difference in activity returns during the delay intervals.
Fig. 5.
Response histograms for a population of 40 PF neurons that had significant sample-selective delay activity. Responses are shown separately for trials in which the “best” stimulus was used as the sample and trials in which the “worst” stimulus was used as the sample. Bin width, 40 msec. The average baseline firing rate was 10 spikes/sec.
To quantify the difference in delay activity after the best and worst samples, we computed an index for each cell by first subtracting the average activity after the worst sample from the activity after the best sample and then dividing the difference by the sum of the two means. The higher the index, the greater the difference in activity after the best and worst samples. Figure 6 shows the distribution of indices for the 40 cells with sample-selective delay activity. Most of the indices cluster around the mean index of 0.19, which corresponds to a 47.7% increase in activity after the best sample over the activity after the worst sample.
Fig. 6.
Distribution of indices showing the difference in delay activity after “best” and “worst” samples for the 40 PF neurons that showed significant sample-selective delay activity. The index is the difference in response to the best and worst sample divided by the sum of the two responses.
To confirm the difference shown in Figure 5, we recomputed the delay activity after the best and worst samples based on just the activity in the second delay interval. The second delay interval was chosen because many PF neurons showed little or no delay activity in the interval immediately after the sample, i.e., in the first delay. As shown in Figure 7A, this test yielded virtually identical results; namely, the differential activity after best and worst samples was retained throughout all delay intervals.
Fig. 7.
Average activity in the delay intervals in PF cortex and IT cortex when the “best” stimulus had been used as the sample and when the “worst” stimulus had been used as the sample. For this figure, “best” and “worst” were determined by the level of activity in the second delay interval. The error bars indicate the SEM. A shows the average delay activity for the 40 PF neurons with sample-selective delay activity; B shows the data for 25 IT neurons with sample-selective delay activity. The average baseline firing rate for the IT neurons was 5.5 spikes/sec.
As an additional test of the ability of PF neurons to convey information about the sample across all delay intervals, we conducted a discriminant analysis on the delay activity of each cell. The discriminant analysis fit normal distributions to the delay activity after each of the six sample stimuli, and then attempted to classify which of the six stimuli had been used as the sample on each trial based on the difference between the delay activity and the means of the six distributions. This analysis does not depend on determining which sample stimulus was “best” or “worst.” To eliminate any optimistic bias in the classification, the discriminant analysis was performed with cross-validation, i.e., the distribution means were computed on half of the data, and these means then were used to classify the stimuli used in the other half of the data (Miller et al., 1993). To test whether the differential delay activity survived intervening stimuli, we included only the delay intervals after at least one intervening stimulus, excluding the delay activity in the interval immediately after the sample.
The discriminant analysis was significant for the same cells that showed a significant effect of SAMPLE (the significance test is virtually identical to that of ANOVA with SAMPLE as factor described above). The mean classification rate for these cells was 20.8%, which was significantly different from chance performance of 16.7% (paired_t_ test, p < 0.001). These results support the conclusion that delay activity in PF cortex conveys a significant amount of information about the sample, even after intervening stimuli.
Comparison of delay activity and stimulus responses
A cell’s preference for particular stimuli during the delay interval was not necessarily the same as its preference during the stimulus intervals. Of the 40 cells with sample-selective delay activity, more than half (23/40, or 58%) responded about equally to all of the stimuli (nonsignificant effect of STIMULUS). For 14 of the cells that did have stimulus-selective responses, we were able to compare the ranking of stimulus preferences during the delay with preferences during the stimulus intervals (for 3 cells, the firing rate was too low for us to be confident of the rankings). For seven of these cells, there was good agreement between the selectivity during the delay and during the stimulus intervals, an example of which is shown in Figure 8A. For the other seven cells, the pattern of selectivity for delay activity and stimulus responses was substantially different, an example of which is shown in Figure8B.
Fig. 8.
Average stimulus responses and delay activity of two PF neurons with sample-selective delay activity. The hatched bars show the average responses to each of the six complex stimuli, and the open bars show the average activity in the delays when those stimuli were used as samples. The error bars indicate the SEM. The rank orders of stimulus responses and delay activity were in good correspondence for the cell illustrated in A, but in poor correspondence for the cell illustrated in B. The baseline firing rate for the cell in A was 10.3 spikes/sec, and for the cell in B 24.3 spikes/sec.
Trends within a delay interval
Although the delay activity of many cells consisted of a relatively constant rate of increased firing throughout the delay interval, other cells showed increasing or decreasing trends in activity. To quantify the changes in activity within a delay interval, we calculated an index for each cell that equaled the difference in activity between the first and second half of the delay interval divided by the sum of the activity in the first and second halves. Figure 9 shows the distribution of the index, which appears to be continuous, as well as examples of four different delay activity profiles. Approximately half of the cells had a index value close to 0.0, indicating a flat delay activity profile (Fig.9C). Some cells showed a gradual increase in activity during the delay, culminating in a transient visual response in the test stimulus interval (Fig. 9A). Other cells showed a sharp inhibition of activity shortly after test stimulus onset followed by a quick recovery and then relatively constant activity for the remainder of the delay (Fig. 9B). Finally, other cells showed a decreasing trend in activity during the delay but an increasing trend during the stimulus presentations (Fig. 9D). For each of the different delay activity profiles, the overall magnitude of activity frequently was sample-selective.
Fig. 9.
Examples of four different profiles of delay activity, taken from four different PF neurons. The top of the figure is a distribution of indices showing the difference in delay activity between the first and second halves of each delay interval for the 82 PF neurons that showed activity in the delays that was significantly above baseline firing rate. The index is the activity in the first half of the delay minus the activity in the second half of the delay divided by their sum. A_–_D, Index values for the four single-cell examples shown in the _bottom_of the figure. The gray bars indicate nonmatch stimulus presentation intervals. The delay intervals illustrated were the intervals immediately after the first nonmatch stimulus in the sequence. Bin width, 10 msec. Baseline firing rates for these cells were 6.4 spikes/sec (A), 12.8 spikes/sec (B), 12 spikes/sec (C), and 5 spikes/sec (D).
Trends across delay intervals
Just as many cells showed increasing or decreasing trends in activity within a delay interval, approximately half the total cells had delay activity that significantly increased or decreased across the multiple delay intervals within a trial (significant INTERVAL factor, described above). Most of these cells (61/82, or 74%) showed an increasing trend; that is, significantly greater activity in the delay intervals later in the trial than in the earlier delays. The pattern of increasing activity varied across cells. For example, the cells illustrated in Figure 10, A and_B_, had little activity in the delay after the sample but a marked increase in activity after the second intervening stimulus. By contrast, the cell illustrated in Figure 10C had a regular increase in delay activity as the trial progressed. The remaining cells (21/82, or 26%) showed a decreasing trend, i.e., greater activity in the early delay intervals than in the later delay intervals. For virtually all of these cells, the drop in delay activity occurred after the first intervening stimulus in the sequence, an example of which is shown in Figure 10D.
Fig. 10.
Examples of three PF neurons with “climbing” delay activity (A_–_C) and a neuron that showed the opposite trend, i.e., “decreasing” delay activity (D). The gray bars indicate stimulus presentation intervals. S, Sample; NM, nonmatch; M, match. Shown are data from trials in which three nonmatches intervened between the sample and final match. Bin width, 40 msec. The baseline firing rates for these neurons were 13.2 spikes/sec (A), 5.1 spikes/sec (B), 13.6 spikes/sec (C), and 10.4 spikes/sec (D).
For some of the cells that had increasing or decreasing trends in their delay activity, the amount of delay activity also was selective for different samples (significant main effects for both INTERVAL and SAMPLE). The incidence of sample-selective delay activity was higher for the cells that showed a decreasing trend (10/21, or 48%) than for the cells that showed an increasing trend (18/61, or 30%), but this difference was not significant (χ2,p = 0.31).
To determine whether the different trends in delay activity represented discrete classes of cells, we calculated for each cell an index that equaled the difference in activity between the first two and last two delay intervals, divided by the sum of the activity in the first two and last two intervals. Similar to what we found for within-delay trends, the index was continuously distributed, without any evidence for discrete classes.
Responses to matching and nonmatching test stimuli
In a previous study of IT neurons, we found that many cells responded differently to test stimuli depending on whether they matched the sample. To test for this in PF cortex, we computed a two-way ANOVA on the responses of each cell to all six test stimuli, with stimulus and match–nonmatching status as factors. This analysis was performed on the 75 PF cells with excitatory visually evoked responses.
The responses of most cells (51/75, or 68%) varied significantly depending on whether the test stimulus was a match or a nonmatch. The majority of these cells (32/51, or 63%) gave stronger responses to test stimuli that matched the sample than when the same stimuli were nonmatching, an effect we will call “match enhancement.” The remaining cells (19/51, or 37%) responded less to matching than nonmatching stimuli, which we will call “match suppression.” The monkey performing the standard task had a somewhat higher proportion of cells with match enhancement (75%, or 12/16, of the cells with match–nonmatch effects) than did the monkey performing the_ABBA_ task (57%, or 20/35), but this difference was not significant (χ2, p = 0.221). Only two cells (2/51, or 4%) showed mixed effects, i.e., suppression to some stimuli and enhancement to others. Thus, enhancement and suppression appear to be mediated by two distinct classes of PF cells.
The average match-enhancement effect was a 74% increase in match responses over nonmatch responses (SE = 2.8%), and the average match suppression effect was a 41% increase in nonmatch responses over match responses (SE = 1.9%). Additional comparisons of the strengths of match enhancement and suppression are described below. The majority of cells with match–nonmatch effects also showed stimulus selectivity in their response to the test stimuli (33/51, or 65%). For these cells, the match–nonmatch status of the stimulus increased or decreased the strengths of the responses without disrupting stimulus selectivity. The incidence of stimulus selectivity for match-enhancement cells (22/32, or 69%) was not significantly different from the incidence for match suppression cells (11/19, or 58%; χ2, p = 0.43). A minority of cells with significant match–nonmatch effects showed no stimulus selectivity (18/51, or 35%). Although this group of cells might, in principle, convey a pure “match–nonmatch” signal, regardless of stimulus, the number of stimuli tested on each cell was too small to be confident that the cells would respond equally to all stimuli.
In a previous study of IT neurons (Miller and Desimone, 1994), we found that cells with match enhancement and suppression differed in how they responded to the repeated nonmatch stimuli on ABBA trials. Cells for which responses to match stimuli were suppressed compared with nonmatching stimuli showed equal suppression for nonmatching stimuli that were repetitions of each other (e.g., the “B B” in_ABBA_), even though the latter stimulus repetitions were behaviorally irrelevant. By contrast, IT cells showing enhancement only gave enhanced responses for the one stimulus that matched the sample.
To examine this issue for PF neurons, we asked whether match enhancement or suppression also extended to the repeated nonmatch stimuli in the monkey performing the ABBA task. To do this, we first determined which stimuli for each cell elicited a significant match–nonmatch effect (determined by a t test applied to the match and nonmatch responses for each stimulus, evaluated at_p_ < 0.05). Of the 245 stimuli that elicited an excitatory visual response, most had significant match–nonmatch effects (192/245, or 78%), with enhancement more common (118/192, or 61%) than suppression (74/192, or 39%). The average responses to stimuli under the match, nonmatch, and repeated nonmatch conditions are shown separately for stimuli with match enhancement and match suppression in Figure 11. The match-suppression results are described below. For the stimuli with match-enhancement effects (Fig. 11A), the responses to the test stimuli in the matching condition were clearly larger than to the same stimuli in the repeated nonmatch conditions. In fact, the average response in the nonmatching and repeated nonmatch conditions was about the same for these stimuli. Thus, as in IT cortex, match enhancement in PF cortex occurred only when a test stimulus matched the sample, which the animal was actively maintaining in working memory. Enhancement was not engaged by simple repetition of the test stimulus (the repeated nonmatch).
Fig. 11.
Average responses across cells to the same set of stimuli appearing as samples and as matches and nonmatches after different numbers of intervening stimuli. Zero intervening stimuli refers to the first test stimulus after the sample in the sequence. The error bars indicate the SEM. A, Average responses to stimuli that elicited match enhancement. B, Average responses to stimuli that elicited match suppression.
The specificity of the enhancement effect for the match condition, and not for the repeated nonmatch, was confirmed by examining response histograms averaged from all stimuli with match enhancement relative to the nonmatch. The histograms illustrated in Figure12A indicate that the average response in the match condition was larger than in either the nonmatch or repeated nonmatch condition. Furthermore, the population average indicates that the enhancement of the match response compared with the nonmatch response began ∼110–120 msec after stimulus onset and well before the animal’s mean behavioral response latency of 376 msec (range, 317–523 msec).
In contrast to the match-enhancement effects, the average response data in Figure 11B show that match suppression occurred in both the match and the behaviorally irrelevant, repeated nonmatch conditions. However, unlike the case for match-suppression effects in IT cortex, the responses in the repeated nonmatch condition were not quite as suppressed as in the match condition. Thus, match suppression in PF cortex appears to be caused largely by simple stimulus repetition, although there may be an additional small suppressive effect caused specifically by matching the behaviorally relevant sample stimulus.
Relationship of properties within cells
The variety of different response properties found in PF cortex raised the question of whether some properties consistently occurred together. To answer this, we examined the distribution of stimulus–response selectivity, sample-selective delay activity, and match–nonmatch effects. The only clear trend was that cells that were nonselective on any one of these three measures also were likely to be nonselective on the other dimensions. For example, cells that failed to show stimulus selectivity in their responses also were unlikely to show sample-selective delay activity (24/104, or 23%) or match–nonmatch effects (20/104, or 19%). In comparison, cells with stimulus-selective responses were more likely to have sample-selective delay activity (16/41, or 39%) and match–nonmatch effects (31/41, or 76%).
Comparisons of PF and IT cortex
We compared the properties of PF neurons with the properties of 135 neurons from the anterior–ventral IT cortex of the monkey performing the ABBA task with the same set of stimuli as used in PF cortex. The IT recording sites in this animal were in perirhinal cortex, between the anterior middle temporal and rhinal sulci (Miller et al., 1993; Miller and Desimone, 1994).
The relative incidences of visual responsiveness, stimulus selectivity, sample-selective delay activity, match enhancement, and match suppression are given in Table 1. All of the values for the IT cells were computed using the same statistical tests that were used on the PF cells, described in previous sections. On average, IT cells were more often visually responsive and more often stimulus-selective than PF cells, which is consistent with the presumably greater role that IT cortex plays in the analysis of visual object features. The minority of PF cells that were stimulus-selective seemed, superficially at least, to have stimulus properties similar to those of IT cells, but we made no attempt to determine any underlying feature selectivity. By comparison, PF cells more often had sample-selective delay activity and match enhancement, which are consistent with a more important role in working memory. The incidence of match suppression was comparable in both areas.
Table 1.
Incidence of effects in PF cortex and IT cortex
An even more striking difference between the two areas was that sample-selective delay activity in IT cortex did not bridge intervening stimuli, unlike in PF cortex. To compare the effects of intervening stimuli on delay activity, we determined separately for each IT and PF cell which sample resulted in the greatest delay activity in the second delay interval (“best” sample), and which sample resulted in the least activity in the same interval (“worst” sample). We then computed the activity in each of the delay intervals separately after the best and worst samples, averaged across all cells with significant sample-selective delay activity, according to the ANOVA. For the PF cells, shown in Figure 7A, the activity after the best sample was higher than that after the worst sample in each of the delay intervals. For the IT cells, shown in Figure 7B, the activity after the best sample was higher than that after the worst in interval two, which was expected, because the responses were sorted on the basis of interval-two activity. However, the activity in the other intervals was not consistently higher for the best sample. Thus, consistent with the results of our previous study in different monkeys, IT neurons appear to be unable to maintain sample-selective delay activity over a time period when the animal is attending to other, physically different stimuli (Miller et al., 1993). This rule may not apply when the stimuli after the sample are identical to the sample itself, because delay activity in the temporal polar cortex appears to be maintained during multiple repetitions of the sample stimulus (Nakamura and Kubota, 1995).
Match enhancement was not only more common in PF cortex than in IT cortex, it also was greater in magnitude. To quantify the magnitude of enhancement and suppression effects, we computed an index for each stimulus by subtracting the mean match response from mean nonmatch response and dividing the absolute value of the difference by the sum of the two means. The higher the index, the stronger the effect (values of 0 indicate equal responses to matches and nonmatches). Figure13A shows the distribution of indices for the stimuli that elicited a significant match-enhancement effect. The median enhancement index for PF cortex was 0.27, compared with only 0.10 in IT cortex. By contrast, the indices for match suppression were similar in the two regions. Figure 13B shows the distribution for stimuli that elicited a significant match-suppression effect. These distributions largely overlap, and the medians of the two distributions are similar (PF cortex: 0.17; IT cortex: 0.12). Thus, match-enhancement effects were stronger in PF than in IT cortex, whereas the strength of the match-suppression effect was similar in the two regions.
Fig. 13.
Distribution of indices showing the strength of the match-enhancement effect (A) and match-suppression effect (B) in PF cortex and IT cortex. The index is the absolute value of the difference between match and nonmatch responses divided by their sum.
We computed a two-way ANOVA on the index values using AREA (IT vs PF cortex) as one factor and ENHANCEMENT–SUPPRESSION as the other factor. This revealed that (1) match–nonmatch effects were larger in PF than in IT cortex (significant effect of AREA, p < 0.001); (2) match-enhancement effects were larger than match-suppression effects (significant effect of ENHANCEMENT–SUPPRESSION, p = 0.02); and (3) there was a significant interaction between the factors (p < 0.001). Similar results were obtained from a discriminant analysis applied to the responses to matching and nonmatching test stimuli.
DISCUSSION
To perform the DMS task, the monkey must be able to both maintain a memory of the sample and evaluate whether a test stimulus matches it. We and others have previously reported a possible neural basis for the latter in IT cortex, in that many IT cells respond differently to test stimuli depending on whether they match the sample (Gross et al., 1979;Mikami and Kubota, 1980; Baylis and Rolls, 1987; Miller et al., 1991b,1993; Riches et al., 1991; Eskandar et al., 1992; Miller and Desimone, 1994, Vogels et al., 1995). We now find that the same type of information about the matching–nonmatching status of the test stimulus is present in PF cortex. In both areas, the effects on the test stimulus responses survive all of the stimuli that intervene between the sample and the match.
There also is a possible neural basis for the sample memory trace in IT and PF cortex, in that cells in both areas have sample-selective delay activity. However, we have found that delay activity in PF cortex is fundamentally different from that in IT cortex, because sample-selective delay activity survives intervening stimuli in the former but not in the latter area. In PF cortex, an explicit neural representation of the sample stimulus appears to be maintained throughout the trial, whereas in IT cortex, it is not. Thus, PF cortex has explicit neural signals correlated with two critical aspects of the DMS task, namely the maintenance of the sample memory trace and an evaluation of whether the test stimulus is a match to it.
Although our findings are most relevant for the “ventral stream,” which underlies object recognition, there is a striking parallel with recent results in the “dorsal stream,” which underlies spatial perception. Neurons in both posterior parietal (PP) cortex and in the more dorsal portion of PF cortex (area 46) have delay activity that is selective for spatial location (Fuster et al., 1982; Kojima and Goldman-Rakic, 1982; Gnadt and Andersen, 1988; Funahashi et al., 1989,1993; di Pellegrino and Wise, 1993a,b). When cells are tested in a spatial version of DMS with multiple intervening stimuli, delay activity in PP cortex does not survive the first intervening stimulus after the sample (Constantinidis and Steinmetz, 1996). By contrast, spatial delay activity in PF cortex is maintained throughout the trial (di Pellegrino and Wise, 1993a,b). An exception to this rule occurs for PP activity immediately preceding a saccade to a target, which is not disrupted by the presentation of a second target in a double-saccade task (Barash et al., 1991a,b; Andersen, 1995).
We found a number of other differences between IT and PF cortex. First, far fewer cells in PF cortex gave stimulus-selective responses than in IT cortex. This is consistent with the idea that PF cortex is more involved in behavioral functions rather than visual recognition or the coding of complex objects per se.
A second difference is that many PF cells had a progressive increase in firing rate as the trial progressed, an effect we have not observed in IT cortex. Similarly, Fuster and colleagues observed PF cells with “climbing” activity across delays without intervening stimuli (Quintana and Fuster, 1992). Climbing activity suggests coding of a future event or action that the monkey expects to occur. This type of memory often is referred to as “prospective memory,” and the animal behavior literature contains abundant evidence for it (seeRoitblat, 1993). In fact, Quintana and Fuster (1992) found that the rate of climbing activity for some PF cells was related to the probability of a forthcoming behavioral response. In our study, the match stimulus at the end of the trial varied according to the sample, but all other events at the end of the trial were the same. We found that for some PF cells, the degree of climbing activity depended on the particular sample used on a trial and, thus, might represent a prospective code for a particular matching test stimulus. For the other cells with nonselective climbing activity, the activity might code any of the other events anticipated at the end of the trial.
A third difference concerns the modulation of responses to test stimuli depending on whether they matched the sample. In IT cortex, approximately half of the cells show such match–nonmatch effects; most of these cells had suppressed responses to matching test stimuli, and the remainder had enhanced responses (Miller et al., 1991b, 1993;Miller and Desimone, 1994). Among visually responsive PF cells, the proportion of cells with significant match–nonmatch effects was greater than in IT cortex. Furthermore, cells with match enhancement were more prevalent in PF than in IT cortex, and they showed a larger difference in response between the match stimulus on one hand, and the nonmatch and repeated nonmatch on the other.
We argued previously that match enhancement and match suppression subserve two different types of short-term memory (Miller and Desimone, 1994). In both IT and PF cortex, match enhancement occurs only for the stimulus that matches the sample, whereas match suppression occurs for any stimulus repetition in the sequence, even if behaviorally irrelevant. In our ABBA version of DMS, for example, match enhancement occurs only for the matching A stimulus, whereas match-suppression occurs equally for the matching A and the irrelevant “repeated nonmatch,” B. In IT cortex, suppressed responses to repeated stimuli also occur both during passive fixation of stimulus sequences and under anesthesia (Miller et al., 1991a; Riches et al., 1991; Vogels et al., 1995). Thus, match enhancement, most prevalent in PF cortex, appears to contribute to active, or working, memory, whereas match suppression, most prevalent in IT cortex, may contribute to the automatic detection of stimulus repetitions.
Interestingly, some of the PF match-enhancement cells were not stimulus-selective; i.e., they gave about equal magnitude-enhanced responses to every matching stimulus tested. This property was very rare in IT cortex, where enhancement effects were nearly always added to an underlying stimulus selectivity. Although the number of stimuli tested on each cell was too small to conclude that PF cells give a pure “match” response, this should be tested in future studies. Taken together, the differences between PF and IT cortex support the idea that PF cortex plays a relatively larger role in working memory.
The neural mechanism of working memory
We have previously proposed a “biased competition” model of attention and working memory in which “top-down” feedback inputs to visual cortex bias responses in favor of stimuli that are actively sought or that currently are relevant to behavior (Desimone et al., 1994, 1995; Desimone and Duncan, 1995; Miller, 1995). In this view, the match enhancement shown by neurons in DMS tasks is caused, in part, by these biasing inputs, which are activated at the time of the sample stimulus presentation and then maintained throughout the trial. We have proposed that these same inputs are responsible for the preferential activation of IT neurons by target stimuli in attention tasks (Chelazzi et al., 1993). The effects of these inputs probably extend to many extrastriate areas besides IT cortex, including area V4, where neuronal responses also are modulated during performance of both attentional and working memory tasks (Moran and Desimone, 1985; Haenny et al., 1988;Maunsell et al., 1991). The top-down inputs presumably arise from areas that are not strictly visual areas themselves, because behavioral relevance often is defined by the task at hand rather than by intrinsic properties of the stimuli.
Several lines of evidence suggest that PF cortex is a major source of the proposed top-down inputs, including the fact that PF cells have sample-selective activity that is maintained for the length of the trial, that PF cortex is heavily interconnected with both IT cortex and other extrastriate visual areas (Ungerleider et al., 1989), and that PF lesions impair performance on DMS tasks with small stimulus sets, which cannot be solved by judgments of novelty or familiarity (Pribram and Mishkin, 1956; Passingham, 1975; Bauer and Fuster, 1976; Mishkin and Manning, 1978). A direct test would be to measure the effects of PF deactivation on match enhancement in IT cortex. This has not yet been tried; however, Fuster et al. (1985) found that IT delay activity became less selective during cooling of PF cortex.
What is the function of delay activity in IT cortex? One possibility is that it maintains a short-term sensory trace of the immediately preceding stimulus, which might play a role in the integration of information over eye movements. However, in separate studies, we found that it sometimes predicts a _following_stimulus, if the monkey expects that stimulus to occur (our unpublished data). In fact, in these studies, we have observed delay activity in IT cortex under all of the same conditions in which one would expect to find it in PF cortex—except after intervening stimuli. Thus, although the presence of delay activity after the sample in IT cortex may serve as a sign, or marker, of biasing inputs from PF cortex, its function at present remains unclear.
Footnotes
- We thank Mortimer Mishkin for his advice and support, Leonardo Chelazzi, who participated in the early phase of the experiment, and James Mazer and Marlene Wicherski for helpful comments on this manuscript.
Correspondence should be addressed to Dr. Robert Desimone, National Institute of Mental Health, 49 Convent Drive, Building 49, Room 1B80, Bethesda, MD 20892-4415.