Peter Dominey - Academia.edu (original) (raw)

Papers by Peter Dominey

Research paper thumbnail of Can We Already Conceive of a Computational Construction Grammar that is Adequate to Address the Data of Neurolinguistics?

Research paper thumbnail of Explaining the neural activity distribution associated with discrete movement sequences: Evidence for parallel functional systems

Cognitive, Affective, & Behavioral Neuroscience, 2018

To explore the effects of practice we scanned participants with fMRI while they were performing f... more To explore the effects of practice we scanned participants with fMRI while they were performing four-key unfamiliar and familiar sequences, and compared the associated activities relative to simple control sequences. On the basis of a recent cognitive model of sequential motor behavior (C-SMB), we propose that the observed neural activity would be associated with three functional networks that can operate in parallel and that allow (a) responding to stimuli in a reaction mode, (b) sequence execution using spatial sequence representations in a central-symbolic mode, and (c) sequence execution using motor chunk representations in a chunking mode. On the basis of this model and findings in the literature, we predicted which neural areas would be active during execution of the unfamiliar and familiar keying sequences. The observed neural activities were largely in line with our predictions, and allowed functions to be attributed to the active brain areas that fit the three above functional systems. The results corroborate C-SMB's assumption that at advanced skill levels the systems executing motor chunks and translating key-specific stimuli are racing to trigger individual responses. They further support recent behavioral indications that spatial sequence representations continue to be used.

Research paper thumbnail of Beyond the word and image: characteristics of a common meaning system for language and vision revealed by functional and structural imaging

NeuroImage, 2015

This research tests the hypothesis that comprehension of human events will engage an extended sem... more This research tests the hypothesis that comprehension of human events will engage an extended semantic representation system, independent of the input modality (sentence vs. picture). To investigate this, we examined brain activation and connectivity in 19 subjects who read sentences and viewed pictures depicting everyday events, in a combined fMRI and DTI study. Conjunction of activity in understanding sentences and pictures revealed a common fronto-temporo-parietal network that included the middle and inferior frontal gyri, the parahippocampal-retrosplenial complex, the anterior and middle temporal gyri, the inferior parietal lobe in particular the temporo-parietal cortex. DTI tractography seeded from this temporo-parietal cortex hub revealed a multi-component network reaching into the temporal pole, the ventral frontal pole and premotor cortex. A significant correlation was found between the relative pathway density issued from the temporo-parietal cortex and the imageability of ...

Research paper thumbnail of Discrete sequence production with and without a pause: the role of cortex, basal ganglia, and cerebellum

Frontiers in human neuroscience, 2013

Our sensorimotor experience unfolds in sequences over time. We hypothesize that the processing of... more Our sensorimotor experience unfolds in sequences over time. We hypothesize that the processing of movement sequences with and without a temporal pause will recruit distinct but cooperating neural processes, including cortico-striatal and cortico-cerebellar networks. We thus, compare neural activity during sequence learning in the presence and absence of this pause. Young volunteer participants learned sensorimotor sequences using the discrete sequence production (DSP) task, with Pause, No-Pause, and Control sequences of four elements in an event related fMRI protocol. The No-Pause and Pause sequences involved a more complex sequential structure than the Control sequence, while the Pause sequences involved insertion of a temporal pause, relative to the No-Pause sequence. The Pause vs. No-Pause contrast revealed extensive fronto-parietal, striatal, thalamic and cerebellar activations, preferentially for the Pause sequences. ROI analysis indicated that the cerebellum displays an early ...

Research paper thumbnail of Evidence for a shared mechanism in linguistic and non-linguistic sequence processing? ERP recordings of online function-and content-information integration

Research paper thumbnail of Multiple Brain Regions Cooperate in Sequential Saccade Generation

Research Notes in Neural Computing, 1991

We model control of voluntary saccades to visual and remembered targets in terms of interactions ... more We model control of voluntary saccades to visual and remembered targets in terms of interactions between the basal ganglia (caudate and substantia nigra), superior colliculus, mediodorsal thalamus, posterior parietal cortex, frontal eye fields and the saccade generator of the brainstem. These interactions include modulation of topographic inhibitory masks that manage motor field activity, dynamic remapping of sensory maps onto the eye motor map to take account of eye movements, and sustained neural activity that embodies spatial memory. Models of these mechanisms implemented in NSL (our Neural Simulation Language) simulate behavior and neural activity described in the literature, and suggest new experiments.

Research paper thumbnail of A Hybrid Propositional-Embodied Cognitive Architecture for Human-Robot Cooperation

2008 Ieee International Joint Conference on Neural Networks, Vols 1-8, 2008

Abstract Robot platforms have now reached a level of technical development wherein they are becom... more Abstract Robot platforms have now reached a level of technical development wherein they are becoming physically capable of useful interaction with humans, while ensuring safety and reasonable cost. The current challenge is for cognitive systems science to provide ...

Research paper thumbnail of Training with cognitive sequences improves syntactic comprehension in agrammatic aphasics

NeuroReport, 2003

A major open question in cognitive neuroscience concerns the modularity of language: does human l... more A major open question in cognitive neuroscience concerns the modularity of language: does human language rely, in part, on neural processes that are not language speci¢c? Such reliance would predict that learning should transfer between non-linguistic and linguistic domains via this common neural basis.To test this prediction, we studied e¡ects of non-linguistic cognitive sequence training on syntactic comprehension of six left-hemisphere damaged aphasic patients. Syntactic comprehension impairment was quanti-¢ed before and after 10 weeks of training on a non-linguistic sequence processing task. This task used a transformational rule speci¢cally corresponding to the transformation required for understanding a particular type of sentence referred to as relativised. Non-linguistic sequencing improved signi¢cantly with training (day e¡ect: F(9,45)¼3.7, po0.005). Moreover, a signi¢cant transfer of this improvement was observed for relativised, but not active nor passive sentences (pre-post  type interaction: F(2,10)¼4.72, po0.05).The speci¢city of this transfer indicates that language relies partially on functional and neural processes that are not language speci¢c. NeuroReport 14 :495^499

Research paper thumbnail of ERP correlates of lexical analysis: N280 reflects processing complexity rather than category or frequency effects

NeuroReport, 2005

In the context of language processing, the N280 is an anterior negative event-related potential p... more In the context of language processing, the N280 is an anterior negative event-related potential pro¢le associated with the lexical categorization of grammatical function words versus content words. Subsequent studies suggested that this e¡ect was related to word statistics including length and frequency in the lexicon.The current research tests the hypothesis that the N280 e¡ect is related to an index of grammatical complexity. We recorded event-related potentials during a sentence reading task. Comparing content versus function words revealed the classic N280. Within function words, we compared the relative pronouns 'qui' and 'que' (which are identical for length and frequency) that in French indicate a subsequent simple (subject^subject) and complex (subject^object) relative clause, respectively. A left anterior N280 e¡ect was observed only for 'que', supporting our hypothesis that the N280 re£ects grammatical complexity that can be confounded with lexical category and statistical properties. NeuroReport16:1435^1438

Research paper thumbnail of Do you agree? Electrophysiological characterization of online agreement checking during the comprehension of correct French passive sentences

Journal of Neurolinguistics, 2007

With this research we investigated the real-time electrophysiological correlates of noun-verb agr... more With this research we investigated the real-time electrophysiological correlates of noun-verb agreement checking during the comprehension of correct passive sentences in French. Event-related potentials were acquired while participants read passive sentences that contained covert (singular, masculine) or overt (plural, feminine) noun-verb agreement. Results show that the processing of overtly or covertly agreeing verbs in passive sentences is associated with an asymmetrical electrophysiological response, reflecting former psycholinguistic evidence of markedness and unmarkedness of certain features. The reading of an overtly marked verb agreeing in number and gender with a feminine plural subject was associated with a left anterior negativity (LAN), whereas covertly marked verbs were associated with a negativity presenting a central-posterior distribution, an N400. These results, confirming the lexical status of features and their immediate but asymmetrical checking during sentence comprehension are discussed in the context of current linguistic and psycholinguistic models of agreement checking.

Research paper thumbnail of Dynamic Construction Grammar and Steps Towards the Narrative Construction of Meaning

Dynamic construction grammar (DCG) is a neurcomputational framework for learning and generalizing... more Dynamic construction grammar (DCG) is a neurcomputational framework for learning and generalizing sentence-tomeaning mappings. It is inspired by the cue competition hypothesis of Bates and MacWhinney, and learns regularities in the ordering of open and closed class words and the corresponding mapping to semantic roles for the open class words. The structure of meaning is a crucial aspect of these form to meaning mappings. Here we describe the DCG framework, and the evolution of meaning representations that have been addressed. The first and most basic meaning representation is a predicate-argument form indicating the predicate, agent, object and recipient. We developed an action recognition system, that detected simple actions and used naïve subjects’ narration to train the model to understand. The DCG comprehension model was then extended to address sentence production. The resulting models were then integrated into a cooperative humanoid robotic platform. We then demonstrated how ...

Research paper thumbnail of Cognitive sequence processing and syntactic comprehension in schizophrenia

NeuroReport, 2000

It has been repeatedly demonstrated that schizophrenic patients are impaired in the comprehension... more It has been repeatedly demonstrated that schizophrenic patients are impaired in the comprehension of sentences with complex syntax. We investigated the hypothesis that this syntactic comprehension impairment in schizophrenia is not a purely linguistic dysfunction, but rather the re¯ection of a cognitive sequence processing impairment that is revealed as task complexity increases. We tested 10 schizophrenic patients using a standard measure of syntactic comprehension, and a non-linguistic sequence processing task, both of which required simple and complex transformation processing. Patients' performance impairment on the two tasks was highly correlated (r 2 0.84), and there was a signi®cant effect for complexity, independent of the task. These results are quite similar to those of aphasic patients with left hemisphere lesions. This suggests that syntactic comprehension de®cits in schizophrenia reveal the dysfunction of cognitive sequence processing mechanisms that can be expressed both in linguistic and nonlinguistic sequence tasks.

Research paper thumbnail of ERP analysis of cognitive sequencing: a left anterior negativity related to structural transformation processing

A major objective of cognitive neuroscience is to identify those neurocomputational processes tha... more A major objective of cognitive neuroscience is to identify those neurocomputational processes that may be shared by multiple cognitive functions vs those that are highly speci®c. This problem of identifying general vs specialized functions is of particular interest in the domain of language processing. Within this domain, event related brain potential (ERP) studies have demonstrated a left anterior negativity (LAN) in a range 300± 700 ms, associated with syntactic processing, often linked to grammatical function words. These words have little or no semantic content, but rather play a role in encoding syntactic structure required for parsing. In the current study we test the hypothesis that the LAN re¯ects the operation of a more general sequence processing capability in which special symbols encode structural information that, when combined with past elements in the sequence, allows the prediction of successor elements. We recorded ERPs during a non-linguistic sequencing task that required subjects (n 10) to process special symbols possessing the functional property de®ned above. When compared to ERPs in a control condition, function symbol processing elicits a left anterior negative shift between temporal and spatial characteristics quite similar to the LAN described during function word processing in language, supporting our hypothesis. These results are discussed in the context of related studies of syntactic and cognitive sequence processing. NeuroReport 11:3187±3191 & 2000 Lippincott

Research paper thumbnail of Cortical and basal ganglia function in temporal segmentation of sensory-motor sequences revealed by event related fMRI

<p&amp... more <p>The serial order and the temporal structure of sensory-motor sequences are tightly coupled, and interact in terms of their neural representation. This has been characterized behaviourally in the discrete sequence processing (DSP) task, in which key-press sequences of 4-6 elements are learned over repeated trials. In this task subjects adopt a temporal segmentation structure in which an initial set of

Research paper thumbnail of Progress in Programming the HRP-2 Humanoid Using Spoken Language

Proceedings 2007 IEEE International Conference on Robotics and Automation, 2007

The current research analyses and demonstrates how spoken language can be used by human users to ... more The current research analyses and demonstrates how spoken language can be used by human users to communicate with the HRP-2 humanoid to program the robot's behavior in a cooperative task. The task involves the humans and the HRP-2 working together to assemble a piece of furniture. The objectives of the system are to 1. Allow the human to impart knowledge of how to accomplish a cooperative task to the robot, i.e. to program the robot, in the form of a sensory-motor action plan. 2. To do this in a seminatural and real-time manner using spoken language. In this framework, a system for Spoken Language Programming (SLP) is presented, and experimental results are presented from this prototype system. In Experiment 1, the human programs the robot to assist in assembling a small table. In Experiment 2, the generalization of the system is demonstrated as the user programs the robot to assist in taking the table apart. The SLP is evaluated in terms of the changes in efficiency as revealed by task completion time and number of command operations required to accomplish the tasks with and without SLP. Lessons learned are discussed, along with plans for improving the system, including developing a richer base of robot action and perception predicates that will allow the use of richer language. We thus demonstrate -for the first time -the capability for a human user to tell a humanoid what to do in a cooperative task so that in real time, the robot performs the task, and acquires new skills that significantly facilitate the cooperative human-robot interaction.

Research paper thumbnail of Effects of subthalamic nucleus stimulation on actual and imagined movement in Parkinson's disease : a PET study

Journal of neurology, 2002

PET studies in moderately affected Parkinson's disease (PD) patients reveal abnormal cerebral... more PET studies in moderately affected Parkinson's disease (PD) patients reveal abnormal cerebral activation during motor execution and imagery, but the effects of subthalamic nucleus (STN) stimulation are not well established. to assess the effect of STN stimulation on cerebral activation during actual and imagined movement in patients with advanced PD. seven severely affected PD patients treated with bilateral STN stimulation were studied with PET and H(2)(15)O. The following conditions were investigated: (1). rest; (2). motor execution of a sequential predefined joystick movement with the right hand and (3). motor imagery of the same task. Patients were studied with and without left STN stimulation while right stimulator remained off. Without STN stimulation, the primary motor cortex was activated only during motor execution whereas the dorsolateral prefrontal cortex (DLPFC) was activated only during motor imagery. An activation of the supplementary motor area (SMA) was seen duri...

Research paper thumbnail of Encoding behavioral context in recurrent networks of the fronto-striatal system: a simulation study

Brain research. Cognitive brain research, 1997

This research addresses the hypothesis that behavioral context is encoded in recurrent networks o... more This research addresses the hypothesis that behavioral context is encoded in recurrent networks of the fronto-striatal system. Behavioral context influences the processing of subsequent brain events, including responses to sensory inputs, thus providing a basis for context-dependent behavior. We define context-dependent behavior as the adaptive ability to produce the appropriate response to a given stimulus, dependent upon the context in which it appears. Behavioral context can change with a time-scale on the order of seconds to tens of seconds or more. This suggests a flexible mechanism that encodes context via an ensemble of neural activation that will appropriately influence the processing of subsequent sensory stimuli. We present a functional model of context encoding in recurrent connections of the fronto-striatal system with simulation results that correspond closely to empirical data. Neuronal activity in monkeys that perform a context-dependent task indicate that the prefron...

Research paper thumbnail of Motor imagery of a lateralized sequential task is asymmetrically slowed in hemi-Parkinson's patients

Research paper thumbnail of Robot command, interrogation and teaching via social interaction

5th IEEE-RAS International Conference on Humanoid Robots, 2005., 2005

The development of high performance robot platforms provides complex systems with which humans mu... more The development of high performance robot platforms provides complex systems with which humans must interact, and levy serious requirements on the quality and depth of these interactions. At the same time, developments in spoken language technology, and in theories of social cognition and intentional cooperative behavior provide the technical basis and theoretical background respectively for the technical specification of how these systems can work.

Research paper thumbnail of Comparison of Classifiers for Decoding Sensory and Cognitive Information from Prefrontal Neuronal Populations

PLoS ONE, 2014

Decoding neuronal information is important in neuroscience, both as a basic means to understand h... more Decoding neuronal information is important in neuroscience, both as a basic means to understand how neuronal activity is related to cerebral function and as a processing stage in driving neuroprosthetic effectors. Here, we compare the readout performance of six commonly used classifiers at decoding two different variables encoded by the spiking activity of the non-human primate frontal eye fields (FEF): the spatial position of a visual cue, and the instructed orientation of the animal's attention. While the first variable is exogenously driven by the environment, the second variable corresponds to the interpretation of the instruction conveyed by the cue; it is endogenously driven and corresponds to the output of internal cognitive operations performed on the visual attributes of the cue. These two variables were decoded using either a regularized optimal linear estimator in its explicit formulation, an optimal linear artificial neural network estimator, a nonlinear artificial neural network estimator, a non-linear naïve Bayesian estimator, a non-linear Reservoir recurrent network classifier or a non-linear Support Vector Machine classifier. Our results suggest that endogenous information such as the orientation of attention can be decoded from the FEF with the same accuracy as exogenous visual information. All classifiers did not behave equally in the face of population size and heterogeneity, the available training and testing trials, the subject's behavior and the temporal structure of the variable of interest. In most situations, the regularized optimal linear estimator and the non-linear Support Vector Machine classifiers outperformed the other tested decoders.

Research paper thumbnail of Can We Already Conceive of a Computational Construction Grammar that is Adequate to Address the Data of Neurolinguistics?

Research paper thumbnail of Explaining the neural activity distribution associated with discrete movement sequences: Evidence for parallel functional systems

Cognitive, Affective, & Behavioral Neuroscience, 2018

To explore the effects of practice we scanned participants with fMRI while they were performing f... more To explore the effects of practice we scanned participants with fMRI while they were performing four-key unfamiliar and familiar sequences, and compared the associated activities relative to simple control sequences. On the basis of a recent cognitive model of sequential motor behavior (C-SMB), we propose that the observed neural activity would be associated with three functional networks that can operate in parallel and that allow (a) responding to stimuli in a reaction mode, (b) sequence execution using spatial sequence representations in a central-symbolic mode, and (c) sequence execution using motor chunk representations in a chunking mode. On the basis of this model and findings in the literature, we predicted which neural areas would be active during execution of the unfamiliar and familiar keying sequences. The observed neural activities were largely in line with our predictions, and allowed functions to be attributed to the active brain areas that fit the three above functional systems. The results corroborate C-SMB's assumption that at advanced skill levels the systems executing motor chunks and translating key-specific stimuli are racing to trigger individual responses. They further support recent behavioral indications that spatial sequence representations continue to be used.

Research paper thumbnail of Beyond the word and image: characteristics of a common meaning system for language and vision revealed by functional and structural imaging

NeuroImage, 2015

This research tests the hypothesis that comprehension of human events will engage an extended sem... more This research tests the hypothesis that comprehension of human events will engage an extended semantic representation system, independent of the input modality (sentence vs. picture). To investigate this, we examined brain activation and connectivity in 19 subjects who read sentences and viewed pictures depicting everyday events, in a combined fMRI and DTI study. Conjunction of activity in understanding sentences and pictures revealed a common fronto-temporo-parietal network that included the middle and inferior frontal gyri, the parahippocampal-retrosplenial complex, the anterior and middle temporal gyri, the inferior parietal lobe in particular the temporo-parietal cortex. DTI tractography seeded from this temporo-parietal cortex hub revealed a multi-component network reaching into the temporal pole, the ventral frontal pole and premotor cortex. A significant correlation was found between the relative pathway density issued from the temporo-parietal cortex and the imageability of ...

Research paper thumbnail of Discrete sequence production with and without a pause: the role of cortex, basal ganglia, and cerebellum

Frontiers in human neuroscience, 2013

Our sensorimotor experience unfolds in sequences over time. We hypothesize that the processing of... more Our sensorimotor experience unfolds in sequences over time. We hypothesize that the processing of movement sequences with and without a temporal pause will recruit distinct but cooperating neural processes, including cortico-striatal and cortico-cerebellar networks. We thus, compare neural activity during sequence learning in the presence and absence of this pause. Young volunteer participants learned sensorimotor sequences using the discrete sequence production (DSP) task, with Pause, No-Pause, and Control sequences of four elements in an event related fMRI protocol. The No-Pause and Pause sequences involved a more complex sequential structure than the Control sequence, while the Pause sequences involved insertion of a temporal pause, relative to the No-Pause sequence. The Pause vs. No-Pause contrast revealed extensive fronto-parietal, striatal, thalamic and cerebellar activations, preferentially for the Pause sequences. ROI analysis indicated that the cerebellum displays an early ...

Research paper thumbnail of Evidence for a shared mechanism in linguistic and non-linguistic sequence processing? ERP recordings of online function-and content-information integration

Research paper thumbnail of Multiple Brain Regions Cooperate in Sequential Saccade Generation

Research Notes in Neural Computing, 1991

We model control of voluntary saccades to visual and remembered targets in terms of interactions ... more We model control of voluntary saccades to visual and remembered targets in terms of interactions between the basal ganglia (caudate and substantia nigra), superior colliculus, mediodorsal thalamus, posterior parietal cortex, frontal eye fields and the saccade generator of the brainstem. These interactions include modulation of topographic inhibitory masks that manage motor field activity, dynamic remapping of sensory maps onto the eye motor map to take account of eye movements, and sustained neural activity that embodies spatial memory. Models of these mechanisms implemented in NSL (our Neural Simulation Language) simulate behavior and neural activity described in the literature, and suggest new experiments.

Research paper thumbnail of A Hybrid Propositional-Embodied Cognitive Architecture for Human-Robot Cooperation

2008 Ieee International Joint Conference on Neural Networks, Vols 1-8, 2008

Abstract Robot platforms have now reached a level of technical development wherein they are becom... more Abstract Robot platforms have now reached a level of technical development wherein they are becoming physically capable of useful interaction with humans, while ensuring safety and reasonable cost. The current challenge is for cognitive systems science to provide ...

Research paper thumbnail of Training with cognitive sequences improves syntactic comprehension in agrammatic aphasics

NeuroReport, 2003

A major open question in cognitive neuroscience concerns the modularity of language: does human l... more A major open question in cognitive neuroscience concerns the modularity of language: does human language rely, in part, on neural processes that are not language speci¢c? Such reliance would predict that learning should transfer between non-linguistic and linguistic domains via this common neural basis.To test this prediction, we studied e¡ects of non-linguistic cognitive sequence training on syntactic comprehension of six left-hemisphere damaged aphasic patients. Syntactic comprehension impairment was quanti-¢ed before and after 10 weeks of training on a non-linguistic sequence processing task. This task used a transformational rule speci¢cally corresponding to the transformation required for understanding a particular type of sentence referred to as relativised. Non-linguistic sequencing improved signi¢cantly with training (day e¡ect: F(9,45)¼3.7, po0.005). Moreover, a signi¢cant transfer of this improvement was observed for relativised, but not active nor passive sentences (pre-post  type interaction: F(2,10)¼4.72, po0.05).The speci¢city of this transfer indicates that language relies partially on functional and neural processes that are not language speci¢c. NeuroReport 14 :495^499

Research paper thumbnail of ERP correlates of lexical analysis: N280 reflects processing complexity rather than category or frequency effects

NeuroReport, 2005

In the context of language processing, the N280 is an anterior negative event-related potential p... more In the context of language processing, the N280 is an anterior negative event-related potential pro¢le associated with the lexical categorization of grammatical function words versus content words. Subsequent studies suggested that this e¡ect was related to word statistics including length and frequency in the lexicon.The current research tests the hypothesis that the N280 e¡ect is related to an index of grammatical complexity. We recorded event-related potentials during a sentence reading task. Comparing content versus function words revealed the classic N280. Within function words, we compared the relative pronouns 'qui' and 'que' (which are identical for length and frequency) that in French indicate a subsequent simple (subject^subject) and complex (subject^object) relative clause, respectively. A left anterior N280 e¡ect was observed only for 'que', supporting our hypothesis that the N280 re£ects grammatical complexity that can be confounded with lexical category and statistical properties. NeuroReport16:1435^1438

Research paper thumbnail of Do you agree? Electrophysiological characterization of online agreement checking during the comprehension of correct French passive sentences

Journal of Neurolinguistics, 2007

With this research we investigated the real-time electrophysiological correlates of noun-verb agr... more With this research we investigated the real-time electrophysiological correlates of noun-verb agreement checking during the comprehension of correct passive sentences in French. Event-related potentials were acquired while participants read passive sentences that contained covert (singular, masculine) or overt (plural, feminine) noun-verb agreement. Results show that the processing of overtly or covertly agreeing verbs in passive sentences is associated with an asymmetrical electrophysiological response, reflecting former psycholinguistic evidence of markedness and unmarkedness of certain features. The reading of an overtly marked verb agreeing in number and gender with a feminine plural subject was associated with a left anterior negativity (LAN), whereas covertly marked verbs were associated with a negativity presenting a central-posterior distribution, an N400. These results, confirming the lexical status of features and their immediate but asymmetrical checking during sentence comprehension are discussed in the context of current linguistic and psycholinguistic models of agreement checking.

Research paper thumbnail of Dynamic Construction Grammar and Steps Towards the Narrative Construction of Meaning

Dynamic construction grammar (DCG) is a neurcomputational framework for learning and generalizing... more Dynamic construction grammar (DCG) is a neurcomputational framework for learning and generalizing sentence-tomeaning mappings. It is inspired by the cue competition hypothesis of Bates and MacWhinney, and learns regularities in the ordering of open and closed class words and the corresponding mapping to semantic roles for the open class words. The structure of meaning is a crucial aspect of these form to meaning mappings. Here we describe the DCG framework, and the evolution of meaning representations that have been addressed. The first and most basic meaning representation is a predicate-argument form indicating the predicate, agent, object and recipient. We developed an action recognition system, that detected simple actions and used naïve subjects’ narration to train the model to understand. The DCG comprehension model was then extended to address sentence production. The resulting models were then integrated into a cooperative humanoid robotic platform. We then demonstrated how ...

Research paper thumbnail of Cognitive sequence processing and syntactic comprehension in schizophrenia

NeuroReport, 2000

It has been repeatedly demonstrated that schizophrenic patients are impaired in the comprehension... more It has been repeatedly demonstrated that schizophrenic patients are impaired in the comprehension of sentences with complex syntax. We investigated the hypothesis that this syntactic comprehension impairment in schizophrenia is not a purely linguistic dysfunction, but rather the re¯ection of a cognitive sequence processing impairment that is revealed as task complexity increases. We tested 10 schizophrenic patients using a standard measure of syntactic comprehension, and a non-linguistic sequence processing task, both of which required simple and complex transformation processing. Patients' performance impairment on the two tasks was highly correlated (r 2 0.84), and there was a signi®cant effect for complexity, independent of the task. These results are quite similar to those of aphasic patients with left hemisphere lesions. This suggests that syntactic comprehension de®cits in schizophrenia reveal the dysfunction of cognitive sequence processing mechanisms that can be expressed both in linguistic and nonlinguistic sequence tasks.

Research paper thumbnail of ERP analysis of cognitive sequencing: a left anterior negativity related to structural transformation processing

A major objective of cognitive neuroscience is to identify those neurocomputational processes tha... more A major objective of cognitive neuroscience is to identify those neurocomputational processes that may be shared by multiple cognitive functions vs those that are highly speci®c. This problem of identifying general vs specialized functions is of particular interest in the domain of language processing. Within this domain, event related brain potential (ERP) studies have demonstrated a left anterior negativity (LAN) in a range 300± 700 ms, associated with syntactic processing, often linked to grammatical function words. These words have little or no semantic content, but rather play a role in encoding syntactic structure required for parsing. In the current study we test the hypothesis that the LAN re¯ects the operation of a more general sequence processing capability in which special symbols encode structural information that, when combined with past elements in the sequence, allows the prediction of successor elements. We recorded ERPs during a non-linguistic sequencing task that required subjects (n 10) to process special symbols possessing the functional property de®ned above. When compared to ERPs in a control condition, function symbol processing elicits a left anterior negative shift between temporal and spatial characteristics quite similar to the LAN described during function word processing in language, supporting our hypothesis. These results are discussed in the context of related studies of syntactic and cognitive sequence processing. NeuroReport 11:3187±3191 & 2000 Lippincott

Research paper thumbnail of Cortical and basal ganglia function in temporal segmentation of sensory-motor sequences revealed by event related fMRI

<p&amp... more <p>The serial order and the temporal structure of sensory-motor sequences are tightly coupled, and interact in terms of their neural representation. This has been characterized behaviourally in the discrete sequence processing (DSP) task, in which key-press sequences of 4-6 elements are learned over repeated trials. In this task subjects adopt a temporal segmentation structure in which an initial set of

Research paper thumbnail of Progress in Programming the HRP-2 Humanoid Using Spoken Language

Proceedings 2007 IEEE International Conference on Robotics and Automation, 2007

The current research analyses and demonstrates how spoken language can be used by human users to ... more The current research analyses and demonstrates how spoken language can be used by human users to communicate with the HRP-2 humanoid to program the robot's behavior in a cooperative task. The task involves the humans and the HRP-2 working together to assemble a piece of furniture. The objectives of the system are to 1. Allow the human to impart knowledge of how to accomplish a cooperative task to the robot, i.e. to program the robot, in the form of a sensory-motor action plan. 2. To do this in a seminatural and real-time manner using spoken language. In this framework, a system for Spoken Language Programming (SLP) is presented, and experimental results are presented from this prototype system. In Experiment 1, the human programs the robot to assist in assembling a small table. In Experiment 2, the generalization of the system is demonstrated as the user programs the robot to assist in taking the table apart. The SLP is evaluated in terms of the changes in efficiency as revealed by task completion time and number of command operations required to accomplish the tasks with and without SLP. Lessons learned are discussed, along with plans for improving the system, including developing a richer base of robot action and perception predicates that will allow the use of richer language. We thus demonstrate -for the first time -the capability for a human user to tell a humanoid what to do in a cooperative task so that in real time, the robot performs the task, and acquires new skills that significantly facilitate the cooperative human-robot interaction.

Research paper thumbnail of Effects of subthalamic nucleus stimulation on actual and imagined movement in Parkinson's disease : a PET study

Journal of neurology, 2002

PET studies in moderately affected Parkinson's disease (PD) patients reveal abnormal cerebral... more PET studies in moderately affected Parkinson's disease (PD) patients reveal abnormal cerebral activation during motor execution and imagery, but the effects of subthalamic nucleus (STN) stimulation are not well established. to assess the effect of STN stimulation on cerebral activation during actual and imagined movement in patients with advanced PD. seven severely affected PD patients treated with bilateral STN stimulation were studied with PET and H(2)(15)O. The following conditions were investigated: (1). rest; (2). motor execution of a sequential predefined joystick movement with the right hand and (3). motor imagery of the same task. Patients were studied with and without left STN stimulation while right stimulator remained off. Without STN stimulation, the primary motor cortex was activated only during motor execution whereas the dorsolateral prefrontal cortex (DLPFC) was activated only during motor imagery. An activation of the supplementary motor area (SMA) was seen duri...

Research paper thumbnail of Encoding behavioral context in recurrent networks of the fronto-striatal system: a simulation study

Brain research. Cognitive brain research, 1997

This research addresses the hypothesis that behavioral context is encoded in recurrent networks o... more This research addresses the hypothesis that behavioral context is encoded in recurrent networks of the fronto-striatal system. Behavioral context influences the processing of subsequent brain events, including responses to sensory inputs, thus providing a basis for context-dependent behavior. We define context-dependent behavior as the adaptive ability to produce the appropriate response to a given stimulus, dependent upon the context in which it appears. Behavioral context can change with a time-scale on the order of seconds to tens of seconds or more. This suggests a flexible mechanism that encodes context via an ensemble of neural activation that will appropriately influence the processing of subsequent sensory stimuli. We present a functional model of context encoding in recurrent connections of the fronto-striatal system with simulation results that correspond closely to empirical data. Neuronal activity in monkeys that perform a context-dependent task indicate that the prefron...

Research paper thumbnail of Motor imagery of a lateralized sequential task is asymmetrically slowed in hemi-Parkinson's patients

Research paper thumbnail of Robot command, interrogation and teaching via social interaction

5th IEEE-RAS International Conference on Humanoid Robots, 2005., 2005

The development of high performance robot platforms provides complex systems with which humans mu... more The development of high performance robot platforms provides complex systems with which humans must interact, and levy serious requirements on the quality and depth of these interactions. At the same time, developments in spoken language technology, and in theories of social cognition and intentional cooperative behavior provide the technical basis and theoretical background respectively for the technical specification of how these systems can work.

Research paper thumbnail of Comparison of Classifiers for Decoding Sensory and Cognitive Information from Prefrontal Neuronal Populations

PLoS ONE, 2014

Decoding neuronal information is important in neuroscience, both as a basic means to understand h... more Decoding neuronal information is important in neuroscience, both as a basic means to understand how neuronal activity is related to cerebral function and as a processing stage in driving neuroprosthetic effectors. Here, we compare the readout performance of six commonly used classifiers at decoding two different variables encoded by the spiking activity of the non-human primate frontal eye fields (FEF): the spatial position of a visual cue, and the instructed orientation of the animal's attention. While the first variable is exogenously driven by the environment, the second variable corresponds to the interpretation of the instruction conveyed by the cue; it is endogenously driven and corresponds to the output of internal cognitive operations performed on the visual attributes of the cue. These two variables were decoded using either a regularized optimal linear estimator in its explicit formulation, an optimal linear artificial neural network estimator, a nonlinear artificial neural network estimator, a non-linear naïve Bayesian estimator, a non-linear Reservoir recurrent network classifier or a non-linear Support Vector Machine classifier. Our results suggest that endogenous information such as the orientation of attention can be decoded from the FEF with the same accuracy as exogenous visual information. All classifiers did not behave equally in the face of population size and heterogeneity, the available training and testing trials, the subject's behavior and the temporal structure of the variable of interest. In most situations, the regularized optimal linear estimator and the non-linear Support Vector Machine classifiers outperformed the other tested decoders.