John Hummel | University of Illinois at Urbana-Champaign (original) (raw)

Papers by John Hummel

Proceedings of the Annual Meeting of the Cognitive Science Society, 2005

Relational reasoning is central in human cognition. Numerous computational models address the com... more Relational reasoning is central in human cognition. Numerous computational models address the component processes of relational reasoning, however these models require the modeler to hand-code the vocabulary of relations on which the model operates. The acquisition of relational concepts remains poorly understood. We present a theory of relation discovery instantiated in a symbolic-connectionist model, which learns structured representations of attributes and relations from unstructured distributed representations of objects by a process of comparison, and subsequently refines these representations through a process of mapping-based schema induction.

bioRxiv (Cold Spring Harbor Laboratory), Dec 15, 2021

Humans are particularly sensitive to changes in the relationships between parts of objects. It re... more Humans are particularly sensitive to changes in the relationships between parts of objects. It remains unclear why this is. One hypothesis is that relational features are highly diagnostic of object categories and emerge as a result of learning to classify objects. We tested this by analysing the internal representations of supervised convolutional neural networks (CNNs) trained to classify large sets of objects. We found that CNNs do not show the same sensitivity to relational changes as previously observed for human participants. Furthermore, when we precisely controlled the deformations to objects, human behaviour was best predicted by the amount of relational changes while CNNs were equally sensitive to all changes. Even changing the statistics of the learning environment by making relations uniquely diagnostic did not make networks more sensitive to relations in general. Our results show that learning to classify objects is not sufficient for the emergence of human shape representations. .

Researchers studying the correspondences between Deep Neural Networks (DNNs) and humans often giv... more Researchers studying the correspondences between Deep Neural Networks (DNNs) and humans often give little consideration to severe testing when drawing conclusions from empirical findings, and this is impeding progress in building better models of minds. We first detail what we mean by severe testing and highlight how this is especially important when working with opaque models with many free parameters that may solve a given task in multiple different ways. Second, we provide multiple examples of researchers making strong claims regarding DNN-human similarities without engaging in severe testing of their hypotheses. Third, we consider why severe testing is undervalued. We provide evidence that part of the fault lies with the review process. There is now a widespread appreciation in many areas of science that a bias for publishing positive results (among other practices) is leading to a credibility crisis, but there seems less awareness of the problem here.

We describe the MindSet benchmark designed to facilitate the testing of DNNs against controlled e... more We describe the MindSet benchmark designed to facilitate the testing of DNNs against controlled experiments reported in psychology. MindSet will focus on a range of low-, middle-, and high-level visual findings that provide important constraints for theory, provide the materials for testing DNNs, and provide an example of how to assess a DNN on each experiment using a ResNet152 pretrained on ImageNet. The goal is not to evaluate how well ResNet152 accounts for human vision, but rather, encourage researchers to assess how well various DNNs account for a range of key human visual phenomena.

Psychological Review

People readily generalize knowledge to novel domains and stimuli. We present a theory, instantiat... more People readily generalize knowledge to novel domains and stimuli. We present a theory, instantiated in a computational model, based on the idea that cross-domain generalization in humans is a case of analogical inference over structured (i.e., symbolic) relational representations. The model is an extension of the LISA (Hummel & Holyoak, 1997) and DORA (Doumas et al., 2008) models of relational inference and learning. The resulting model learns both the content and format (i.e., structure) of relational representations from non-relational inputs without supervision, when augmented with the capacity for reinforcement learning, leverages these representations to learn individual domains, and then generalizes to new domains on the first exposure (i.e., zero-shot learning) via analogical inference. We demonstrate the capacity of the model to learn structured relational representations from a variety of simple visual stimuli, and to perform cross-domain generalization between video games (Breakout and Pong) and between several psychological tasks. We demonstrate that the model's trajectory closely mirrors the trajectory of children as they learn about relations, accounting for phenomena from the literature on the development of children's reasoning and analogy making. The model's ability to generalize between domains demonstrates the flexibility afforded by representing domains in terms of their underlying relational structure, rather than simply in terms of the statistical relations between their inputs and outputs.

Journal of Experimental Psychology: Learning, Memory and Cognition, 2009

Object images are identified more efficiently after prior exposure. Here, the authors investigate... more Object images are identified more efficiently after prior exposure. Here, the authors investigated shape representations supporting object priming. The dependent measure in all experiments was the minimum exposure duration required to correctly identify an object image in a rapid serial visual presentation stream. Priming was defined as the change in minimum exposure duration for identification as a function of prior exposure to an object. Experiment 1 demonstrated that this dependent measure yielded an estimate of predominantly visual priming (i.e., free of name and concept priming). Experiments 2 and 3 demonstrated that although priming was sensitive to orientation, visual priming was relatively invariant with image inversion (i.e., an image visually primed its inverted counterpart approximately as much as it primed itself). Experiment 4 demonstrated a similar dissociation with images rotated 90°off the upright. In all experiments, the difference in the magnitude of priming for identical or rotated-inverted priming conditions was marginal or nonexistent. These results suggest that visual representations that support priming can be relatively insensitive to picture-plane manipulations, although these manipulations have a substantial effect on object identification.

Canadian journal of psychology, Jun 1, 1992

Phenomenologically, human shape recognition appears to be invariant with changes of orientation i... more Phenomenologically, human shape recognition appears to be invariant with changes of orientation in depth (up to parts occlusion), position in the visual field, and size. Recent versions of template theories (e.g., Ullman, 1989; Lowe, 1987) assume that these invariances are achieved through the application of transformations such as rotation, translation, and scaling of the image so that it can be matched metrically to a stored template. Presumably, such transformations would require time for their execution. We describe recent priming experiments in which the effects of a prior brief presentation of an image on its subsequent recognition are assessed. The results of these experiments indicate that the invariance is complete: The magnitude of visual priming (as distinct from name or basic level concept priming) is not affected by a change in position, size, orientation in depth, or the particular lines and vertices present in the image, as long as representations of the same components can be activated. An implemented seven layer neural network model (Hummel & Bicderman, 1992) that captures these fundamental properties of human object recognition is described. Given a line drawing of an object, the model activates a viewpoint-invariant structural description of the object, specifying its parts and their interrelations. Visual priming is interpreted as a change in the connection weights for the activation of: a) cells, termed geon feature assemblies (GFAs), that conjoin the output of units that represent invariant, independent properties of a single geon and its relations (such as its type, aspect ratio, relations to other geons), or b) a change in the connection weights by which several GFAs activate a cell representing an object. Resume Sur le plan phlnomenologiquc, il scmble que I'etre humain identifie les formes de facon invariante en depit de changements d'orientation en profondcur (jusqu'a la dissimulation de parties), de position dans le champ visucl ct dc grandeur. Selon de recentes versions des theories des patrons (p. ex. Ullman, 1989; Lowe, 1987), il y a in variance dc la forme lorsque des transformations sont effectuees, par exemple unc rotation, unc translation ou encore une mise a I'echelle de I'imagc dc sortc que cclle-ci peut etre assoctee de facon mdtriquc a un patron stockc en memoirc. Dc tcllcs transformations n£cessiteraient vraiscmbla

Toward an Integrated Account of Reflexive and Reflective Reasoning John E. Hummel (jhummel@lifesc... more Toward an Integrated Account of Reflexive and Reflective Reasoning John E. Hummel (jhummel@lifesci.ucla.edu) Department of Psychology University of California Los Angeles 405 Hilgard Ave. Los Angeles, CA 90095-1563 Jesse M. Choplin (choplin@lifesci.ucla.edu) Department of Psychology University of California Los Angeles 405 Hilgard Ave. Los Angeles, CA 90095-1563 Abstract Some inferences are seemingly automatic (reflexive; Shastri & Ajjanagadde, 1993), whereas others require more effort (i.e., are reflective). We present the beginnings of an integrated account of reflexive and reflective reasoning, based on the LISA model of analogical reasoning (Hummel & Holyoak, 1997). The account holds that reflexive inferences are those that can be generated automatically based on existing knowledge in long-term memory, whereas reflective inferences require explicit structure- mapping and therefore demand greater attention and working memory. According to this account, reflexive inferences manife...

Attention, perception & psychophysics, Dec 31, 2013

Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to ... more Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to be roughly three to five items: three to five objects (i.e., bound collections of object features) in the literature on visual WM or three to five role bindings (i.e., objects in specific relational roles) in the literature on memory and reasoning. Three experiments investigated the capacity of observers' WM for the spatial relations among objects in a visual display, and the results suggest that the "items" in WM are neither simply objects nor simply role bindings. The results of Experiment 1 are most consistent with a model that treats an "item" in visual WM as an object, along with the roles of all its relations to one other object. Experiment 2 compared observers' WM for object size with their memory for relative size and provided evidence that observers compute and store objects' relations per se (rather than just absolute size) in WM. Experiment 3 tested and confirmed several more nuanced predictions of the model supported by Experiment 1. Together, these findings suggest that objects are stored in visual WM in pairs (along with all the relations between the objects in a pair) and that, from the perspective of WM, a given object in one pair is not the same "item" as that same object in a different pair. Keywords Visual relations. Spatial relations. Visual working memory. Objects. Role bindings Working memory (WM) is the cognitive resource responsible for the active maintenance and manipulation of information. The capacity of WM, which is sharply limited, determines how much information one can maintain and manipulatethat is, how much one can perceive or think about-in parallel. As such, the capacity of WM is a key bottleneck in perception (

Attention, perception & psychophysics, Dec 31, 2013

Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to ... more Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to be roughly three to five items: three to five objects (i.e., bound collections of object features) in the literature on visual WM or three to five role bindings (i.e., objects in specific relational roles) in the literature on memory and reasoning. Three experiments investigated the capacity of observers' WM for the spatial relations among objects in a visual display, and the results suggest that the "items" in WM are neither simply objects nor simply role bindings. The results of Experiment 1 are most consistent with a model that treats an "item" in visual WM as an object, along with the roles of all its relations to one other object. Experiment 2 compared observers' WM for object size with their memory for relative size and provided evidence that observers compute and store objects' relations per se (rather than just absolute size) in WM. Experiment 3 tested and confirmed several more nuanced predictions of the model supported by Experiment 1. Together, these findings suggest that objects are stored in visual WM in pairs (along with all the relations between the objects in a pair) and that, from the perspective of WM, a given object in one pair is not the same "item" as that same object in a different pair. Keywords Visual relations. Spatial relations. Visual working memory. Objects. Role bindings Working memory (WM) is the cognitive resource responsible for the active maintenance and manipulation of information. The capacity of WM, which is sharply limited, determines how much information one can maintain and manipulatethat is, how much one can perceive or think about-in parallel. As such, the capacity of WM is a key bottleneck in perception (

Proceedings of the National Academy of Sciences, 2015

Journal of Cognitive Neuroscience, 2004

Analogy is important for learning and discovery and is considered a core component of intelligenc... more Analogy is important for learning and discovery and is considered a core component of intelligence. We present a computational account of analogical reasoning that is compatible with data we have collected from patients with cortical degeneration of either their frontal or anterior temporal cortices due to frontotemporal lobar degeneration (FTLD). These two patient groups showed different deficits in picture and verbal analogies: frontal lobe FTLD patients tended to make errors due to impairments in working memory and inhibitory abilities, whereas temporal lobe FTLD patients tended to make errors due to semantic memory loss. Using the “Learning and Inference with Schemas and Analogies” model, we provide a specific account of how such deficits may arise within neural networks supporting analogical problem solving.

Psychology and Aging, 2004

The difficulty of reasoning tasks depends on their relational complexity, which increases with th... more The difficulty of reasoning tasks depends on their relational complexity, which increases with the number of relations that must be considered simultaneously to make an inference, and on the number of irrelevant items that must be inhibited. The authors examined the ability of younger and older adults to integrate multiple relations and inhibit irrelevant stimuli. Young adults performed well at all but the highest level of relational complexity, whereas older adults performed poorly even at a medium level of relational complexity, especially when irrelevant information was presented. Simulations based on a neurocomputational model of analogical reasoning, Learning and Inference with Schemas and Analogies (LISA), suggest that the observed decline in reasoning performance may be explained by a decline in attention and inhibitory functions in older adults.

Psychology and Aging, 2004

The difficulty of reasoning tasks depends on their relational complexity, which increases with th... more The difficulty of reasoning tasks depends on their relational complexity, which increases with the number of relations that must be considered simultaneously to make an inference, and on the number of irrelevant items that must be inhibited. The authors examined the ability of younger and older adults to integrate multiple relations and inhibit irrelevant stimuli. Young adults performed well at all but the highest level of relational complexity, whereas older adults performed poorly even at a medium level of relational complexity, especially when irrelevant information was presented. Simulations based on a neurocomputational model of analogical reasoning, Learning and Inference with Schemas and Analogies (LISA), suggest that the observed decline in reasoning performance may be explained by a decline in attention and inhibitory functions in older adults.

Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images o... more Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images of objects and are often described as the best models of biological vision. This conclusion is largely based on three sets of findings: (1) DNNs are more accurate than any other model in classifying images taken from various datasets, (2) DNNs do the best job in predicting the pattern of human errors in classifying objects taken from various behavioral benchmark datasets, and (3) DNNs do the best job in predicting brain signals in response to images taken from various brain benchmark datasets (e.g., single cell responses or fMRI data). However, most behavioral and brain benchmarks report the outcomes of observational experiments that do not manipulate any independent variables, and we show that the good prediction on these datasets may be mediated by DNNs that share little overlap with biological vision. More problematically, we show that DNNs account for almost no results from psychologi...

Relations play a central role in human perception and cognition, but little is known about how re... more Relations play a central role in human perception and cognition, but little is known about how relational concepts are acquired and predicated. For example, how do we come to understand that physical force is a higher-order multiplicative relation between mass and acceleration? We report an experiment demonstrating that structure mapping (a.k.a., analogical mapping) plays a key role in the predication of novel higher-order relations. This finding suggests that structure mapping-i.e., the appreciation of analogies-may play a pivotal role in the acquisition and predication of novel relational concepts.

To a young child, it may not be immediately obvious that Bill's selling Mary his car implies that... more To a young child, it may not be immediately obvious that Bill's selling Mary his car implies that she now owns the car; but after a sufficient number of examples, the child will eventually induce a schema that makes the relationship between buying and owning reflexive (if evidenced only by the fact that the inference is reflexive for an adult).

Relational reasoning is central in human cognition. Numerous computational models address the com... more Relational reasoning is central in human cognition. Numerous computational models address the component processes of relational reasoning, however these models require the modeler to hand-code the vocabulary of relations on which the model operates. The acquisition of relational concepts remains poorly understood. We present a theory of relation discovery instantiated in a symbolic-connectionist model, which learns structured representations of attributes and relations from unstructured distributed representations of objects by a process of comparison, and subsequently refines these representations through a process of mapping-based schema induction.

Proceedings of the Annual Meeting of the Cognitive Science Society, 2005

Relational reasoning is central in human cognition. Numerous computational models address the com... more Relational reasoning is central in human cognition. Numerous computational models address the component processes of relational reasoning, however these models require the modeler to hand-code the vocabulary of relations on which the model operates. The acquisition of relational concepts remains poorly understood. We present a theory of relation discovery instantiated in a symbolic-connectionist model, which learns structured representations of attributes and relations from unstructured distributed representations of objects by a process of comparison, and subsequently refines these representations through a process of mapping-based schema induction.

bioRxiv (Cold Spring Harbor Laboratory), Dec 15, 2021

Humans are particularly sensitive to changes in the relationships between parts of objects. It re... more Humans are particularly sensitive to changes in the relationships between parts of objects. It remains unclear why this is. One hypothesis is that relational features are highly diagnostic of object categories and emerge as a result of learning to classify objects. We tested this by analysing the internal representations of supervised convolutional neural networks (CNNs) trained to classify large sets of objects. We found that CNNs do not show the same sensitivity to relational changes as previously observed for human participants. Furthermore, when we precisely controlled the deformations to objects, human behaviour was best predicted by the amount of relational changes while CNNs were equally sensitive to all changes. Even changing the statistics of the learning environment by making relations uniquely diagnostic did not make networks more sensitive to relations in general. Our results show that learning to classify objects is not sufficient for the emergence of human shape representations. .

Researchers studying the correspondences between Deep Neural Networks (DNNs) and humans often giv... more Researchers studying the correspondences between Deep Neural Networks (DNNs) and humans often give little consideration to severe testing when drawing conclusions from empirical findings, and this is impeding progress in building better models of minds. We first detail what we mean by severe testing and highlight how this is especially important when working with opaque models with many free parameters that may solve a given task in multiple different ways. Second, we provide multiple examples of researchers making strong claims regarding DNN-human similarities without engaging in severe testing of their hypotheses. Third, we consider why severe testing is undervalued. We provide evidence that part of the fault lies with the review process. There is now a widespread appreciation in many areas of science that a bias for publishing positive results (among other practices) is leading to a credibility crisis, but there seems less awareness of the problem here.

We describe the MindSet benchmark designed to facilitate the testing of DNNs against controlled e... more We describe the MindSet benchmark designed to facilitate the testing of DNNs against controlled experiments reported in psychology. MindSet will focus on a range of low-, middle-, and high-level visual findings that provide important constraints for theory, provide the materials for testing DNNs, and provide an example of how to assess a DNN on each experiment using a ResNet152 pretrained on ImageNet. The goal is not to evaluate how well ResNet152 accounts for human vision, but rather, encourage researchers to assess how well various DNNs account for a range of key human visual phenomena.

Psychological Review

People readily generalize knowledge to novel domains and stimuli. We present a theory, instantiat... more People readily generalize knowledge to novel domains and stimuli. We present a theory, instantiated in a computational model, based on the idea that cross-domain generalization in humans is a case of analogical inference over structured (i.e., symbolic) relational representations. The model is an extension of the LISA (Hummel & Holyoak, 1997) and DORA (Doumas et al., 2008) models of relational inference and learning. The resulting model learns both the content and format (i.e., structure) of relational representations from non-relational inputs without supervision, when augmented with the capacity for reinforcement learning, leverages these representations to learn individual domains, and then generalizes to new domains on the first exposure (i.e., zero-shot learning) via analogical inference. We demonstrate the capacity of the model to learn structured relational representations from a variety of simple visual stimuli, and to perform cross-domain generalization between video games (Breakout and Pong) and between several psychological tasks. We demonstrate that the model's trajectory closely mirrors the trajectory of children as they learn about relations, accounting for phenomena from the literature on the development of children's reasoning and analogy making. The model's ability to generalize between domains demonstrates the flexibility afforded by representing domains in terms of their underlying relational structure, rather than simply in terms of the statistical relations between their inputs and outputs.

Journal of Experimental Psychology: Learning, Memory and Cognition, 2009

Object images are identified more efficiently after prior exposure. Here, the authors investigate... more Object images are identified more efficiently after prior exposure. Here, the authors investigated shape representations supporting object priming. The dependent measure in all experiments was the minimum exposure duration required to correctly identify an object image in a rapid serial visual presentation stream. Priming was defined as the change in minimum exposure duration for identification as a function of prior exposure to an object. Experiment 1 demonstrated that this dependent measure yielded an estimate of predominantly visual priming (i.e., free of name and concept priming). Experiments 2 and 3 demonstrated that although priming was sensitive to orientation, visual priming was relatively invariant with image inversion (i.e., an image visually primed its inverted counterpart approximately as much as it primed itself). Experiment 4 demonstrated a similar dissociation with images rotated 90°off the upright. In all experiments, the difference in the magnitude of priming for identical or rotated-inverted priming conditions was marginal or nonexistent. These results suggest that visual representations that support priming can be relatively insensitive to picture-plane manipulations, although these manipulations have a substantial effect on object identification.

Canadian journal of psychology, Jun 1, 1992

Phenomenologically, human shape recognition appears to be invariant with changes of orientation i... more Phenomenologically, human shape recognition appears to be invariant with changes of orientation in depth (up to parts occlusion), position in the visual field, and size. Recent versions of template theories (e.g., Ullman, 1989; Lowe, 1987) assume that these invariances are achieved through the application of transformations such as rotation, translation, and scaling of the image so that it can be matched metrically to a stored template. Presumably, such transformations would require time for their execution. We describe recent priming experiments in which the effects of a prior brief presentation of an image on its subsequent recognition are assessed. The results of these experiments indicate that the invariance is complete: The magnitude of visual priming (as distinct from name or basic level concept priming) is not affected by a change in position, size, orientation in depth, or the particular lines and vertices present in the image, as long as representations of the same components can be activated. An implemented seven layer neural network model (Hummel & Bicderman, 1992) that captures these fundamental properties of human object recognition is described. Given a line drawing of an object, the model activates a viewpoint-invariant structural description of the object, specifying its parts and their interrelations. Visual priming is interpreted as a change in the connection weights for the activation of: a) cells, termed geon feature assemblies (GFAs), that conjoin the output of units that represent invariant, independent properties of a single geon and its relations (such as its type, aspect ratio, relations to other geons), or b) a change in the connection weights by which several GFAs activate a cell representing an object. Resume Sur le plan phlnomenologiquc, il scmble que I'etre humain identifie les formes de facon invariante en depit de changements d'orientation en profondcur (jusqu'a la dissimulation de parties), de position dans le champ visucl ct dc grandeur. Selon de recentes versions des theories des patrons (p. ex. Ullman, 1989; Lowe, 1987), il y a in variance dc la forme lorsque des transformations sont effectuees, par exemple unc rotation, unc translation ou encore une mise a I'echelle de I'imagc dc sortc que cclle-ci peut etre assoctee de facon mdtriquc a un patron stockc en memoirc. Dc tcllcs transformations n£cessiteraient vraiscmbla

Toward an Integrated Account of Reflexive and Reflective Reasoning John E. Hummel (jhummel@lifesc... more Toward an Integrated Account of Reflexive and Reflective Reasoning John E. Hummel (jhummel@lifesci.ucla.edu) Department of Psychology University of California Los Angeles 405 Hilgard Ave. Los Angeles, CA 90095-1563 Jesse M. Choplin (choplin@lifesci.ucla.edu) Department of Psychology University of California Los Angeles 405 Hilgard Ave. Los Angeles, CA 90095-1563 Abstract Some inferences are seemingly automatic (reflexive; Shastri & Ajjanagadde, 1993), whereas others require more effort (i.e., are reflective). We present the beginnings of an integrated account of reflexive and reflective reasoning, based on the LISA model of analogical reasoning (Hummel & Holyoak, 1997). The account holds that reflexive inferences are those that can be generated automatically based on existing knowledge in long-term memory, whereas reflective inferences require explicit structure- mapping and therefore demand greater attention and working memory. According to this account, reflexive inferences manife...

Attention, perception & psychophysics, Dec 31, 2013

Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to ... more Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to be roughly three to five items: three to five objects (i.e., bound collections of object features) in the literature on visual WM or three to five role bindings (i.e., objects in specific relational roles) in the literature on memory and reasoning. Three experiments investigated the capacity of observers' WM for the spatial relations among objects in a visual display, and the results suggest that the "items" in WM are neither simply objects nor simply role bindings. The results of Experiment 1 are most consistent with a model that treats an "item" in visual WM as an object, along with the roles of all its relations to one other object. Experiment 2 compared observers' WM for object size with their memory for relative size and provided evidence that observers compute and store objects' relations per se (rather than just absolute size) in WM. Experiment 3 tested and confirmed several more nuanced predictions of the model supported by Experiment 1. Together, these findings suggest that objects are stored in visual WM in pairs (along with all the relations between the objects in a pair) and that, from the perspective of WM, a given object in one pair is not the same "item" as that same object in a different pair. Keywords Visual relations. Spatial relations. Visual working memory. Objects. Role bindings Working memory (WM) is the cognitive resource responsible for the active maintenance and manipulation of information. The capacity of WM, which is sharply limited, determines how much information one can maintain and manipulatethat is, how much one can perceive or think about-in parallel. As such, the capacity of WM is a key bottleneck in perception (

Attention, perception & psychophysics, Dec 31, 2013

Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to ... more Across many areas of study in cognition, the capacity of working memory (WM) is widely agreed to be roughly three to five items: three to five objects (i.e., bound collections of object features) in the literature on visual WM or three to five role bindings (i.e., objects in specific relational roles) in the literature on memory and reasoning. Three experiments investigated the capacity of observers' WM for the spatial relations among objects in a visual display, and the results suggest that the "items" in WM are neither simply objects nor simply role bindings. The results of Experiment 1 are most consistent with a model that treats an "item" in visual WM as an object, along with the roles of all its relations to one other object. Experiment 2 compared observers' WM for object size with their memory for relative size and provided evidence that observers compute and store objects' relations per se (rather than just absolute size) in WM. Experiment 3 tested and confirmed several more nuanced predictions of the model supported by Experiment 1. Together, these findings suggest that objects are stored in visual WM in pairs (along with all the relations between the objects in a pair) and that, from the perspective of WM, a given object in one pair is not the same "item" as that same object in a different pair. Keywords Visual relations. Spatial relations. Visual working memory. Objects. Role bindings Working memory (WM) is the cognitive resource responsible for the active maintenance and manipulation of information. The capacity of WM, which is sharply limited, determines how much information one can maintain and manipulatethat is, how much one can perceive or think about-in parallel. As such, the capacity of WM is a key bottleneck in perception (

Proceedings of the National Academy of Sciences, 2015

Journal of Cognitive Neuroscience, 2004

Analogy is important for learning and discovery and is considered a core component of intelligenc... more Analogy is important for learning and discovery and is considered a core component of intelligence. We present a computational account of analogical reasoning that is compatible with data we have collected from patients with cortical degeneration of either their frontal or anterior temporal cortices due to frontotemporal lobar degeneration (FTLD). These two patient groups showed different deficits in picture and verbal analogies: frontal lobe FTLD patients tended to make errors due to impairments in working memory and inhibitory abilities, whereas temporal lobe FTLD patients tended to make errors due to semantic memory loss. Using the “Learning and Inference with Schemas and Analogies” model, we provide a specific account of how such deficits may arise within neural networks supporting analogical problem solving.

Psychology and Aging, 2004

The difficulty of reasoning tasks depends on their relational complexity, which increases with th... more The difficulty of reasoning tasks depends on their relational complexity, which increases with the number of relations that must be considered simultaneously to make an inference, and on the number of irrelevant items that must be inhibited. The authors examined the ability of younger and older adults to integrate multiple relations and inhibit irrelevant stimuli. Young adults performed well at all but the highest level of relational complexity, whereas older adults performed poorly even at a medium level of relational complexity, especially when irrelevant information was presented. Simulations based on a neurocomputational model of analogical reasoning, Learning and Inference with Schemas and Analogies (LISA), suggest that the observed decline in reasoning performance may be explained by a decline in attention and inhibitory functions in older adults.

Psychology and Aging, 2004

The difficulty of reasoning tasks depends on their relational complexity, which increases with th... more The difficulty of reasoning tasks depends on their relational complexity, which increases with the number of relations that must be considered simultaneously to make an inference, and on the number of irrelevant items that must be inhibited. The authors examined the ability of younger and older adults to integrate multiple relations and inhibit irrelevant stimuli. Young adults performed well at all but the highest level of relational complexity, whereas older adults performed poorly even at a medium level of relational complexity, especially when irrelevant information was presented. Simulations based on a neurocomputational model of analogical reasoning, Learning and Inference with Schemas and Analogies (LISA), suggest that the observed decline in reasoning performance may be explained by a decline in attention and inhibitory functions in older adults.

Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images o... more Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images of objects and are often described as the best models of biological vision. This conclusion is largely based on three sets of findings: (1) DNNs are more accurate than any other model in classifying images taken from various datasets, (2) DNNs do the best job in predicting the pattern of human errors in classifying objects taken from various behavioral benchmark datasets, and (3) DNNs do the best job in predicting brain signals in response to images taken from various brain benchmark datasets (e.g., single cell responses or fMRI data). However, most behavioral and brain benchmarks report the outcomes of observational experiments that do not manipulate any independent variables, and we show that the good prediction on these datasets may be mediated by DNNs that share little overlap with biological vision. More problematically, we show that DNNs account for almost no results from psychologi...

Relations play a central role in human perception and cognition, but little is known about how re... more Relations play a central role in human perception and cognition, but little is known about how relational concepts are acquired and predicated. For example, how do we come to understand that physical force is a higher-order multiplicative relation between mass and acceleration? We report an experiment demonstrating that structure mapping (a.k.a., analogical mapping) plays a key role in the predication of novel higher-order relations. This finding suggests that structure mapping-i.e., the appreciation of analogies-may play a pivotal role in the acquisition and predication of novel relational concepts.

To a young child, it may not be immediately obvious that Bill's selling Mary his car implies that... more To a young child, it may not be immediately obvious that Bill's selling Mary his car implies that she now owns the car; but after a sufficient number of examples, the child will eventually induce a schema that makes the relationship between buying and owning reflexive (if evidenced only by the fact that the inference is reflexive for an adult).

Relational reasoning is central in human cognition. Numerous computational models address the com... more Relational reasoning is central in human cognition. Numerous computational models address the component processes of relational reasoning, however these models require the modeler to hand-code the vocabulary of relations on which the model operates. The acquisition of relational concepts remains poorly understood. We present a theory of relation discovery instantiated in a symbolic-connectionist model, which learns structured representations of attributes and relations from unstructured distributed representations of objects by a process of comparison, and subsequently refines these representations through a process of mapping-based schema induction.