Jean-David BOUCHER | Institut polytechnique de Grenoble - Grenoble INP (original) (raw)
Papers by Jean-David BOUCHER
Experimental brain research, Apr 2, 2024
Frontiers in Neurorobotics, 2012
Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid ... more Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should now be able to manipulate and exploit these social cues in cooperation with their human partners. Previous studies have demonstrated that people follow human and robot gaze, and that it can help them to cope with spatially ambiguous language. Our goal is to extend these findings into the domain of action, to determine how human and robot gaze can influence the speed and accuracy of human action. We report on results from a human-human cooperation experiment demonstrating that an agent's vision of her/his partner's gaze can significantly improve that agent's performance in a cooperative task. We then implement a heuristic capability to generate such gaze cues by a humanoid robot that engages in the same cooperative interaction. The subsequent human-robot experiments demonstrate that a human agent can indeed exploit the predictive gaze of their robot partner in a cooperative task. This allows us to render the humanoid robot more human-like in its ability to communicate with humans. The long term objectives of the work are thus to identify social cooperation cues, and to validate their pertinence through implementation in a cooperative robot. The current research provides the robot with the capability to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks. Gaze is manipulated in three conditions: Full gaze (coordinated eye and head), eyes hidden with sunglasses, and head fixed. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.
In previous research, we developed an integrated platform that combined visual scene interpretati... more In previous research, we developed an integrated platform that combined visual scene interpretation with speech processing to provide input to a language learning model. The system was demonstrated to learn a rich set of sentence-meaning mappings that could allow it to construct the appropriate meanings for new sentences in a generalization task. While this demonstrated potential promise, it fell short in several aspects of providing a useful human-robot interaction system. The current research addresses three of these shortcomings, demonstrating the natural extensibility of the platform architecture. First, the system must be able not only to understand what it hears, but also to describe what it sees and to interact with the human user. This is a natural extension of the knowledge of sentence-to-meaning mappings that is now applied in the inverse scene-to-sentence sense. Secondly, we extend the system's ontology from physical events to include spatial relations. We will show that spatial relations are naturally accommodated in the predicate argument representations for events. Finally, because the robot community is international the robot should be able to speak multiple languages, and we thus demonstrate that the language model extends naturally to include both English and Japanese. Concrete results from a working interactive system are presented and future directions for adaptive humanrobot interaction systems are outlined.
The current research presents an original model allowing a machine to acquire new behaviors via i... more The current research presents an original model allowing a machine to acquire new behaviors via its cooperative interaction with a human user. One of specificities of this system is to place the interaction at the heart of the learning. Thus, as one proceeds with exchanges, the robot improves its behaviors favoring a smoother and more natural interaction. Two experiments demonstrate
Lecture Notes in Computer Science, 2006
The current research provides results from three experiments on the ability of a mobile robot to ... more The current research provides results from three experiments on the ability of a mobile robot to acquire new behaviors based on the integration of guidance from a human user and its own internal representation of the resulting perceptual and motor events. The robot learns to associate perceptual state changes with the conditional initiation and cessation of primitive motor behaviors. After several training trials, the system learns to ignore irrelevant perceptual factors, resulting in a robust representation of complex behaviors that require conditional execution based on dynamically changing perceptual states. Three experiments demonstrate the robustness of this approach in learning composite perceptual-motor behavioral sequences of varying complexity.
Human interaction in natural environments relies on a variety of perceptual cues to guide and sta... more Human interaction in natural environments relies on a variety of perceptual cues to guide and stabilize the interaction. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should be able to manipulate and exploit these communicative cues in cooperation with their human partners. In the current research we identify a set of principal communicative speech and gaze cues in human-human interaction, and then formalize and implement these cues in a humanoid robot. The objective of the work is to render the humanoid robot more human-like in its ability to communicate with humans. The first phase of this research, described here, is to provide the robot with a generative capability-that is to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks.. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.
doi: 10.3389/fnbot.2012.00003 I reach faster when I see you look: gaze effects in human–human and... more doi: 10.3389/fnbot.2012.00003 I reach faster when I see you look: gaze effects in human–human and human–robot face-to-face cooperation
The current research presents a system that learns to understand object names, spatial rela-tion ... more The current research presents a system that learns to understand object names, spatial rela-tion terms and event descriptions from observing narrated action sequences. The system extracts meaning from observed visual scenes by exploiting perceptual primitives related to motion and contact in order to represent events and spatial relations as predicate-argument structures. Learning the mapping between sentences and the predicate-argument representations of the situations they describe results in the development of a small lexicon, and a structured set of sentence form–to–meaning mappings, or simplified grammatical constructions. The acquired grammatical construction knowledge general-izes, allowing the system to correctly understand new sentences not used in training. In the context of discourse, the grammatical constructions are used in the inverse sense to generate sentences from meanings, allowing the system to describe visual scenes that it perceives. In question and answer dialo...
One of the defining characteristics of human cognition is our outstanding capacity to cooperate. ... more One of the defining characteristics of human cognition is our outstanding capacity to cooperate. A central requirement for cooperation is the ability to establish a “shared plan ” – which defines the interlaced actions of the two cooperating agents – in real time, and even to negotiate this shared plan during its execution. In the current research we identify the requirements for cooperation, extending our earlier work in this area. These requirements include the ability to negotiate a shared plan using spoken language, to learn new component actions within that plan, based on visual observation and kinesthetic demonstration, and finally to coordinate all of these functions in real time. We present a cognitive system that implements these requirements, and demonstrate the system’s ability to allow a Nao humanoid robot to learn a non-trivial cooperative task in real-time. We further provide a concrete demonstration of how the real-time learning capability can be easily deployed on di...
The objective of this research is to develop a system for language learning based on a minimum of... more The objective of this research is to develop a system for language learning based on a minimum of pre-wired language-specific functionality, that is compatible with observations of perceptual and language capabilities in the human developmental trajectory. In the proposed system, meaning (in terms of descriptions of events and spatial relations) is extracted from video images based on detection of position, motion, physical contact and their parameters. Mapping of sentence form to meaning is performed by learning grammatical constructions that are retrieved from a construction inventory based on the constellation of closed class items uniquely identifying the target sentence structure. The resulting system displays robust acquisition behavior that reproduces certain observations from developmental studies, with very modest “innate” language specificity. Most importantly, the demonstrates a certain degree of autonomy in adapting to the structural regularities of the environment.
Cette these consiste a realiser un programme qui permet a un robot, via une interaction fluide av... more Cette these consiste a realiser un programme qui permet a un robot, via une interaction fluide avec un utilisateur, d'apprendre de nouvelles connaissances qui peuvent etre reutilisees. L'approche adoptee integre le paradigme de l'enaction (autonomie, creation de sens, emergence, incarnation, experience subjective), un savoir intersubjectif (ou connaissance mutuelle), et le formalisme des modeles statistiques d'induction. Par une programmation par demonstration (PbD) et un enseignement kinesthesique, l'utilisateur manie le robot et peut lui apprendre des comportements (mouvements de pattes, de tete etc. ) synchrones ou paralleles, cycliques ou acycliques. Ainsi, avec un minimum d'a priori, le systeme fait emerger des symboles a partir de la segmentation et la detection de regularites dans des flux de donnees sensori-motrices continues. Symboles ou comportements que l'utilisateur labellise et peut reutiliser.
4th IEEE/RAS International Conference on Humanoid Robots, 2004., 2000
In previous research, we developed an integrated platform that combined visual scene interpretati... more In previous research, we developed an integrated platform that combined visual scene interpretation with speech processing to provide input to a language-learning model. The system was demonstrated to learn a rich set of sentence-meaning mappings that could allow it to construct the appropriate meanings for new sentences in a generalization task. While this demonstrated potential promise, it fell short in several aspects of providing a useful human-robot interaction system. The current research addresses three of these shortcomings, demonstrating the natural extensibility of the platform architecture. First, the system must be able not only to understand what it hears, but also to describe what it sees and to interact with the human user. This is a natural extension of the knowledge of sentence-to-meaning mappings that is now applied in the inverse scene-to-sentence sense. Secondly, we extend the system's ontology from physical events to include spatial relations. We show that spatial relations are naturally accommodated in the predicate argument representations for events. Finally, because the robot community is international the robot should be able to speak multiple languages, we thus demonstrate that the language model extends naturally to include both English and Japanese. Concrete results from a working interactive system are presented and future directions for adaptive human-robot interaction systems are outlined.
Robots should be capable of interacting in a cooperative and adaptive manner with their human cou... more Robots should be capable of interacting in a cooperative and adaptive manner with their human counterparts in open-ended tasks that can change in real-time. An important aspect of the robot behavior will be the ability to acquire new knowledge of the cooperative tasks by observing and interacting with humans. The current research addresses this challenge. We present results from a cooperative humanrobot interaction system that has been specifically developed for portability between different humanoid platforms, by abstraction layers at the perceptual and motor interfaces. In the perceptual domain, the resulting system is demonstrated to learn to recognize objects and to recognize actions as sequences of perceptual primitives, and to transfer this learning, and recognition, between different robotic platforms. For execution, composite actions and plans are shown to be learnt on one robot and executed successfully on a different one. Most importantly, the system provides the ability to link actions into shared plans, that form the basis of human-robot cooperation, applying principles from human cognitive development to the domain of robot cognitive systems.
IEEE Transactions on Autonomous Mental Development, 2013
One of the defining characteristics of human cognition is our outstanding capacity to cooperate. ... more One of the defining characteristics of human cognition is our outstanding capacity to cooperate. A central requirement for cooperation is the ability to establish a "shared plan"-which defines the interlaced actions of the two cooperating agents-in real time, and even to negotiate this shared plan during its execution. In the current research we identify the requirements for cooperation, extending our earlier work in this area. These requirements include the ability to negotiate a shared plan using spoken language, to learn new component actions within that plan, based on visual observation and kinesthetic demonstration, and finally to coordinate all of these functions in real time. We present a cognitive system that implements these requirements, and demonstrate the system's ability to allow a Nao humanoid robot to learn a non-trivial cooperative task in real-time. We further provide a concrete demonstration of how the real-time learning capability can be easily deployed on different platform, in this case the iCub humanoid. The results are considered in the context of how the development of language in the human infant provides a powerful lever in the development of cooperative plans from lower-level sensorimotor capabilities.
Proceedings of the 3rd international workshop on Affective interaction in natural environments - AFFINE '10, 2010
Human interaction in natural environments relies on a variety of perceptual cues to guide and sta... more Human interaction in natural environments relies on a variety of perceptual cues to guide and stabilize the interaction. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should be able to manipulate and exploit these communicative cues in cooperation with their human partners. In the current research we identify a set of principal communicative speech and gaze cues in human-human interaction, and then formalize and implement these cues in a humanoid robot. The objective of the work is to render the humanoid robot more human-like in its ability to communicate with humans. The first phase of this research, described here, is to provide the robot with a generative capability-that is to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks.. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.
Lecture Notes in Computer Science, 2006
The current research provides results from three experiments on the ability of a mobile robot to ... more The current research provides results from three experiments on the ability of a mobile robot to acquire new behaviors based on the integration of guidance from a human user and its own internal representation of the resulting perceptual and motor events. The robot learns to associate perceptual state changes with the conditional initiation and cessation of primitive motor behaviors. After several training trials, the system learns to ignore irrelevant perceptual factors, resulting in a robust representation of complex behaviors that require conditional execution based on dynamically changing perceptual states. Three experiments demonstrate the robustness of this approach in learning composite perceptual-motor behavioral sequences of varying complexity.
2006 6th IEEE-RAS International Conference on Humanoid Robots, 2006
The current research presents an original model allowing a machine to acquire new behaviors via i... more The current research presents an original model allowing a machine to acquire new behaviors via its cooperative interaction with a human user. One of specificities of this system is to place the interaction at the heart of the learning. Thus, as one proceeds with exchanges, the robot improves its behaviors favoring a smoother and more natural interaction. Two experiments demonstrate
Frontiers in Neurorobotics, 2012
Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid ... more Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should now be able to manipulate and exploit these social cues in cooperation with their human partners. Previous studies have demonstrated that people follow human and robot gaze, and that it can help them to cope with spatially ambiguous language. Our goal is to extend these findings into the domain of action, to determine how human and robot gaze can influence the speed and accuracy of human action. We report on results from a human-human cooperation experiment demonstrating that an agent's vision of her/his partner's gaze can significantly improve that agent's performance in a cooperative task. We then implement a heuristic capability to generate such gaze cues by a humanoid robot that engages in the same cooperative interaction. The subsequent human-robot experiments demonstrate that a human agent can indeed exploit the predictive gaze of their robot partner in a cooperative task. This allows us to render the humanoid robot more human-like in its ability to communicate with humans. The long term objectives of the work are thus to identify social cooperation cues, and to validate their pertinence through implementation in a cooperative robot. The current research provides the robot with the capability to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks. Gaze is manipulated in three conditions: Full gaze (coordinated eye and head), eyes hidden with sunglasses, and head fixed. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.
Cognitive Systems Research, 2005
The objective of this research is to develop a system for language learning based on a minimum of... more The objective of this research is to develop a system for language learning based on a minimum of pre-wired language-specific functionality, that is compatible with observations of perceptual and language capabilities in the human developmental trajectory. In the proposed system, meaning (in terms of descriptions of events and spatial relations) is extracted from video images based on detection of position, motion, physical contact and their parameters. Mapping of sentence form to meaning is performed by learning grammatical constructions that are retrieved from a construction inventory based on the constellation of closed class items uniquely identifying the target sentence structure. The resulting system displays robust acquisition behavior that reproduces certain observations from developmental studies, with very modest "innate" language specificity.
Experimental brain research, Apr 2, 2024
Frontiers in Neurorobotics, 2012
Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid ... more Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should now be able to manipulate and exploit these social cues in cooperation with their human partners. Previous studies have demonstrated that people follow human and robot gaze, and that it can help them to cope with spatially ambiguous language. Our goal is to extend these findings into the domain of action, to determine how human and robot gaze can influence the speed and accuracy of human action. We report on results from a human-human cooperation experiment demonstrating that an agent's vision of her/his partner's gaze can significantly improve that agent's performance in a cooperative task. We then implement a heuristic capability to generate such gaze cues by a humanoid robot that engages in the same cooperative interaction. The subsequent human-robot experiments demonstrate that a human agent can indeed exploit the predictive gaze of their robot partner in a cooperative task. This allows us to render the humanoid robot more human-like in its ability to communicate with humans. The long term objectives of the work are thus to identify social cooperation cues, and to validate their pertinence through implementation in a cooperative robot. The current research provides the robot with the capability to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks. Gaze is manipulated in three conditions: Full gaze (coordinated eye and head), eyes hidden with sunglasses, and head fixed. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.
In previous research, we developed an integrated platform that combined visual scene interpretati... more In previous research, we developed an integrated platform that combined visual scene interpretation with speech processing to provide input to a language learning model. The system was demonstrated to learn a rich set of sentence-meaning mappings that could allow it to construct the appropriate meanings for new sentences in a generalization task. While this demonstrated potential promise, it fell short in several aspects of providing a useful human-robot interaction system. The current research addresses three of these shortcomings, demonstrating the natural extensibility of the platform architecture. First, the system must be able not only to understand what it hears, but also to describe what it sees and to interact with the human user. This is a natural extension of the knowledge of sentence-to-meaning mappings that is now applied in the inverse scene-to-sentence sense. Secondly, we extend the system's ontology from physical events to include spatial relations. We will show that spatial relations are naturally accommodated in the predicate argument representations for events. Finally, because the robot community is international the robot should be able to speak multiple languages, and we thus demonstrate that the language model extends naturally to include both English and Japanese. Concrete results from a working interactive system are presented and future directions for adaptive humanrobot interaction systems are outlined.
The current research presents an original model allowing a machine to acquire new behaviors via i... more The current research presents an original model allowing a machine to acquire new behaviors via its cooperative interaction with a human user. One of specificities of this system is to place the interaction at the heart of the learning. Thus, as one proceeds with exchanges, the robot improves its behaviors favoring a smoother and more natural interaction. Two experiments demonstrate
Lecture Notes in Computer Science, 2006
The current research provides results from three experiments on the ability of a mobile robot to ... more The current research provides results from three experiments on the ability of a mobile robot to acquire new behaviors based on the integration of guidance from a human user and its own internal representation of the resulting perceptual and motor events. The robot learns to associate perceptual state changes with the conditional initiation and cessation of primitive motor behaviors. After several training trials, the system learns to ignore irrelevant perceptual factors, resulting in a robust representation of complex behaviors that require conditional execution based on dynamically changing perceptual states. Three experiments demonstrate the robustness of this approach in learning composite perceptual-motor behavioral sequences of varying complexity.
Human interaction in natural environments relies on a variety of perceptual cues to guide and sta... more Human interaction in natural environments relies on a variety of perceptual cues to guide and stabilize the interaction. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should be able to manipulate and exploit these communicative cues in cooperation with their human partners. In the current research we identify a set of principal communicative speech and gaze cues in human-human interaction, and then formalize and implement these cues in a humanoid robot. The objective of the work is to render the humanoid robot more human-like in its ability to communicate with humans. The first phase of this research, described here, is to provide the robot with a generative capability-that is to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks.. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.
doi: 10.3389/fnbot.2012.00003 I reach faster when I see you look: gaze effects in human–human and... more doi: 10.3389/fnbot.2012.00003 I reach faster when I see you look: gaze effects in human–human and human–robot face-to-face cooperation
The current research presents a system that learns to understand object names, spatial rela-tion ... more The current research presents a system that learns to understand object names, spatial rela-tion terms and event descriptions from observing narrated action sequences. The system extracts meaning from observed visual scenes by exploiting perceptual primitives related to motion and contact in order to represent events and spatial relations as predicate-argument structures. Learning the mapping between sentences and the predicate-argument representations of the situations they describe results in the development of a small lexicon, and a structured set of sentence form–to–meaning mappings, or simplified grammatical constructions. The acquired grammatical construction knowledge general-izes, allowing the system to correctly understand new sentences not used in training. In the context of discourse, the grammatical constructions are used in the inverse sense to generate sentences from meanings, allowing the system to describe visual scenes that it perceives. In question and answer dialo...
One of the defining characteristics of human cognition is our outstanding capacity to cooperate. ... more One of the defining characteristics of human cognition is our outstanding capacity to cooperate. A central requirement for cooperation is the ability to establish a “shared plan ” – which defines the interlaced actions of the two cooperating agents – in real time, and even to negotiate this shared plan during its execution. In the current research we identify the requirements for cooperation, extending our earlier work in this area. These requirements include the ability to negotiate a shared plan using spoken language, to learn new component actions within that plan, based on visual observation and kinesthetic demonstration, and finally to coordinate all of these functions in real time. We present a cognitive system that implements these requirements, and demonstrate the system’s ability to allow a Nao humanoid robot to learn a non-trivial cooperative task in real-time. We further provide a concrete demonstration of how the real-time learning capability can be easily deployed on di...
The objective of this research is to develop a system for language learning based on a minimum of... more The objective of this research is to develop a system for language learning based on a minimum of pre-wired language-specific functionality, that is compatible with observations of perceptual and language capabilities in the human developmental trajectory. In the proposed system, meaning (in terms of descriptions of events and spatial relations) is extracted from video images based on detection of position, motion, physical contact and their parameters. Mapping of sentence form to meaning is performed by learning grammatical constructions that are retrieved from a construction inventory based on the constellation of closed class items uniquely identifying the target sentence structure. The resulting system displays robust acquisition behavior that reproduces certain observations from developmental studies, with very modest “innate” language specificity. Most importantly, the demonstrates a certain degree of autonomy in adapting to the structural regularities of the environment.
Cette these consiste a realiser un programme qui permet a un robot, via une interaction fluide av... more Cette these consiste a realiser un programme qui permet a un robot, via une interaction fluide avec un utilisateur, d'apprendre de nouvelles connaissances qui peuvent etre reutilisees. L'approche adoptee integre le paradigme de l'enaction (autonomie, creation de sens, emergence, incarnation, experience subjective), un savoir intersubjectif (ou connaissance mutuelle), et le formalisme des modeles statistiques d'induction. Par une programmation par demonstration (PbD) et un enseignement kinesthesique, l'utilisateur manie le robot et peut lui apprendre des comportements (mouvements de pattes, de tete etc. ) synchrones ou paralleles, cycliques ou acycliques. Ainsi, avec un minimum d'a priori, le systeme fait emerger des symboles a partir de la segmentation et la detection de regularites dans des flux de donnees sensori-motrices continues. Symboles ou comportements que l'utilisateur labellise et peut reutiliser.
4th IEEE/RAS International Conference on Humanoid Robots, 2004., 2000
In previous research, we developed an integrated platform that combined visual scene interpretati... more In previous research, we developed an integrated platform that combined visual scene interpretation with speech processing to provide input to a language-learning model. The system was demonstrated to learn a rich set of sentence-meaning mappings that could allow it to construct the appropriate meanings for new sentences in a generalization task. While this demonstrated potential promise, it fell short in several aspects of providing a useful human-robot interaction system. The current research addresses three of these shortcomings, demonstrating the natural extensibility of the platform architecture. First, the system must be able not only to understand what it hears, but also to describe what it sees and to interact with the human user. This is a natural extension of the knowledge of sentence-to-meaning mappings that is now applied in the inverse scene-to-sentence sense. Secondly, we extend the system's ontology from physical events to include spatial relations. We show that spatial relations are naturally accommodated in the predicate argument representations for events. Finally, because the robot community is international the robot should be able to speak multiple languages, we thus demonstrate that the language model extends naturally to include both English and Japanese. Concrete results from a working interactive system are presented and future directions for adaptive human-robot interaction systems are outlined.
Robots should be capable of interacting in a cooperative and adaptive manner with their human cou... more Robots should be capable of interacting in a cooperative and adaptive manner with their human counterparts in open-ended tasks that can change in real-time. An important aspect of the robot behavior will be the ability to acquire new knowledge of the cooperative tasks by observing and interacting with humans. The current research addresses this challenge. We present results from a cooperative humanrobot interaction system that has been specifically developed for portability between different humanoid platforms, by abstraction layers at the perceptual and motor interfaces. In the perceptual domain, the resulting system is demonstrated to learn to recognize objects and to recognize actions as sequences of perceptual primitives, and to transfer this learning, and recognition, between different robotic platforms. For execution, composite actions and plans are shown to be learnt on one robot and executed successfully on a different one. Most importantly, the system provides the ability to link actions into shared plans, that form the basis of human-robot cooperation, applying principles from human cognitive development to the domain of robot cognitive systems.
IEEE Transactions on Autonomous Mental Development, 2013
One of the defining characteristics of human cognition is our outstanding capacity to cooperate. ... more One of the defining characteristics of human cognition is our outstanding capacity to cooperate. A central requirement for cooperation is the ability to establish a "shared plan"-which defines the interlaced actions of the two cooperating agents-in real time, and even to negotiate this shared plan during its execution. In the current research we identify the requirements for cooperation, extending our earlier work in this area. These requirements include the ability to negotiate a shared plan using spoken language, to learn new component actions within that plan, based on visual observation and kinesthetic demonstration, and finally to coordinate all of these functions in real time. We present a cognitive system that implements these requirements, and demonstrate the system's ability to allow a Nao humanoid robot to learn a non-trivial cooperative task in real-time. We further provide a concrete demonstration of how the real-time learning capability can be easily deployed on different platform, in this case the iCub humanoid. The results are considered in the context of how the development of language in the human infant provides a powerful lever in the development of cooperative plans from lower-level sensorimotor capabilities.
Proceedings of the 3rd international workshop on Affective interaction in natural environments - AFFINE '10, 2010
Human interaction in natural environments relies on a variety of perceptual cues to guide and sta... more Human interaction in natural environments relies on a variety of perceptual cues to guide and stabilize the interaction. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should be able to manipulate and exploit these communicative cues in cooperation with their human partners. In the current research we identify a set of principal communicative speech and gaze cues in human-human interaction, and then formalize and implement these cues in a humanoid robot. The objective of the work is to render the humanoid robot more human-like in its ability to communicate with humans. The first phase of this research, described here, is to provide the robot with a generative capability-that is to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks.. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.
Lecture Notes in Computer Science, 2006
The current research provides results from three experiments on the ability of a mobile robot to ... more The current research provides results from three experiments on the ability of a mobile robot to acquire new behaviors based on the integration of guidance from a human user and its own internal representation of the resulting perceptual and motor events. The robot learns to associate perceptual state changes with the conditional initiation and cessation of primitive motor behaviors. After several training trials, the system learns to ignore irrelevant perceptual factors, resulting in a robust representation of complex behaviors that require conditional execution based on dynamically changing perceptual states. Three experiments demonstrate the robustness of this approach in learning composite perceptual-motor behavioral sequences of varying complexity.
2006 6th IEEE-RAS International Conference on Humanoid Robots, 2006
The current research presents an original model allowing a machine to acquire new behaviors via i... more The current research presents an original model allowing a machine to acquire new behaviors via its cooperative interaction with a human user. One of specificities of this system is to place the interaction at the heart of the learning. Thus, as one proceeds with exchanges, the robot improves its behaviors favoring a smoother and more natural interaction. Two experiments demonstrate
Frontiers in Neurorobotics, 2012
Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid ... more Human-human interaction in natural environments relies on a variety of perceptual cues. Humanoid robots are becoming increasingly refined in their sensorimotor capabilities, and thus should now be able to manipulate and exploit these social cues in cooperation with their human partners. Previous studies have demonstrated that people follow human and robot gaze, and that it can help them to cope with spatially ambiguous language. Our goal is to extend these findings into the domain of action, to determine how human and robot gaze can influence the speed and accuracy of human action. We report on results from a human-human cooperation experiment demonstrating that an agent's vision of her/his partner's gaze can significantly improve that agent's performance in a cooperative task. We then implement a heuristic capability to generate such gaze cues by a humanoid robot that engages in the same cooperative interaction. The subsequent human-robot experiments demonstrate that a human agent can indeed exploit the predictive gaze of their robot partner in a cooperative task. This allows us to render the humanoid robot more human-like in its ability to communicate with humans. The long term objectives of the work are thus to identify social cooperation cues, and to validate their pertinence through implementation in a cooperative robot. The current research provides the robot with the capability to produce appropriate speech and gaze cues in the context of human-robot cooperation tasks. Gaze is manipulated in three conditions: Full gaze (coordinated eye and head), eyes hidden with sunglasses, and head fixed. We demonstrate the pertinence of these cues in terms of statistical measures of action times for humans in the context of a cooperative task, as gaze significantly facilitates cooperation as measured by human response times.
Cognitive Systems Research, 2005
The objective of this research is to develop a system for language learning based on a minimum of... more The objective of this research is to develop a system for language learning based on a minimum of pre-wired language-specific functionality, that is compatible with observations of perceptual and language capabilities in the human developmental trajectory. In the proposed system, meaning (in terms of descriptions of events and spatial relations) is extracted from video images based on detection of position, motion, physical contact and their parameters. Mapping of sentence form to meaning is performed by learning grammatical constructions that are retrieved from a construction inventory based on the constellation of closed class items uniquely identifying the target sentence structure. The resulting system displays robust acquisition behavior that reproduces certain observations from developmental studies, with very modest "innate" language specificity.