Amir A L Y [Few Selected Papers Only] | Plymouth University (original) (raw)

Uploads

Papers by Amir A L Y **[Few Selected Papers Only]**

Research paper thumbnail of Social cognitive systems in smart environments: Approaches for learning, reasoning, and adaptation

Cognitive Systems Research, 2019

Robots with social competencies are no longer confinedto controlled laboratory environments as res... more Robots with social competencies are no longer confinedto controlled laboratory environments as research prototypes. They are beginning to appear in the real world as informational guides in airports, museums, and hospitals, collaborating closely alongside humans on factor floors, and assisting families in the home. As robots move into these new spaces, they must grapple with challenging and dynamic environments, requiring them to learn and adapt over time to unforeseen situations. The most important features of these environments are the humans that these robots need to work with and alongside, requiring them to​ learn and adapt specifically to these humans’ cognitive and behavioral characteristics.

Research paper thumbnail of Evaluation of Word Representations in Grounding Natural Language Instructions through Computational Human-Robot Interaction

Proceedings of the 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2019

Abstract—In order to interact with people in a natural way, a robot must be able to link words to... more Abstract—In order to interact with people in a natural way, a robot must be able to link words to objects and actions. Although previous studies in the literature have investigated grounding, they did not consider grounding of unknown synonyms. In this paper, we introduce a probabilistic model for grounding unknown synonymous object and action names using cross-situational learning. The proposed Bayesian learning model uses four different word representations to determine synonymous words. Afterwards, they are grounded through geometric characteristics of objects and kinematic features of the robot joints during action execution. The proposed model is evaluated through an interaction experiment between a human tutor and HSR robot. The results show that semantic and syntactic information both enable grounding of unknown synonyms and that the combination of both achieves the best grounding.

Research paper thumbnail of A Probabilistic Approach to Unsupervised Induction of Combinatory Categorial Grammar in Situated Human-Robot Interaction

IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) Beijing, China, November 6-9, 2018, 2018

Abstract—Robots are progressively moving into spaces that have been primarily shaped by human age... more Abstract—Robots are progressively moving into spaces that have been primarily shaped by human agency; they collaborate with human users in different tasks that require them to understand human language so as to behave appropriately in space. To this end, a stubborn challenge that we address in this paper is inferring the syntactic structure of language,
which embraces grounding parts of speech (e.g., nouns, verbs, and prepositions) through visual perception, and induction of Combinatory Categorial Grammar (CCG) in situated human-robot interaction. This could pave the way towards making a robot able to understand the syntactic relationships between words (i.e., understand phrases), and consequently the meaning of human instructions during interaction, which is a future scope of this current study.

Research paper thumbnail of A Probabilistic Framework for Comparing Syntactic and Semantic Grounding of Synonyms through Cross-Situational Learning

Conference: ICRA-2018 Workshop on "Representing a Complex World: Perception, Inference, and Learning for Joint Semantic, Geometric, and Physical Understanding", Brisbane, Australia, 2018

Natural human-robot interaction requires robots to link words to objects and actions through grou... more Natural human-robot interaction requires robots to link words to objects and actions through grounding. Although grounding has been investigated in previous studies, none of them considered grounding of synonyms. In this paper, we try to fill this gap by introducing a Bayesian learning model for grounding synonymous object and action names using cross-situational learning. Three different word representations are employed with the probabilistic model and evaluated according to their grounding performance. Words are grounded through geometric characteristics of objects and kinematic features of the robot joints during action execution. An interaction experiment between a human tutor and HSR robot is used to evaluate the proposed model. The results show that representing words by syntactic and/or semantic information achieves worse grounding results than representing them by unique numbers.

Research paper thumbnail of Towards Understanding Object-Directed Actions: A Generative Model for Grounding Syntactic Categories of Speech through Visual Perception

Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Australia, 2018

Creating successful human-robot collaboration requires robots to have high-level cognitive functi... more Creating successful human-robot collaboration requires robots to have high-level cognitive functions that could allow them to understand human language and actions in space. To meet this target, an elusive challenge that we address in this paper is to understand object-directed actions through grounding language based on visual cues representing the dynamics of human actions on objects, object characteristics (color and geometry), and spatial relationships between objects in a tabletop scene. The proposed probabilistic framework investigates unsupervised Part-of-Speech (POS) tagging to determine syntactic categories of words so as to infer grammatical structure of language. The dynamics of object-directed actions are characterized through the locations of the human arm joints-modeled on a Hidden Markov Model (HMM)-while manipulating objects, in addition to those of objects represented in 3D point clouds. These corresponding point clouds to segmented objects encode geometric features and spatial semantics of referents and landmarks in the environment. The proposed Bayesian learning model is successfully evaluated through interaction experiments between a human user and Toyota HSR robot in space.

Research paper thumbnail of A Generative Framework for Multimodal Learning of Spatial Concepts and Object Categories: An Unsupervised Part-of-Speech Tagging and 3D Visual Perception Based Approach

Proceedings of the 7th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EPIROB), Portugal, 2017

Future human-robot collaboration employs language in instructing a robot about specific tasks to ... more Future human-robot collaboration employs language in instructing a robot about specific tasks to perform in its surroundings. This requires the robot to be able to associate spatial knowledge with language to understand the details of an assigned task so as to behave appropriately in the context of interaction. In this paper, we propose a probabilistic framework for learning the meaning of language spatial concepts (spatial prepositions) and object categories based on visual cues representing spatial layouts and geometric characteristics of objects in a tabletop scene. The model investigates unsupervised Part-of-Speech (POS) tagging through a Hidden Markov Model (HMM) that infers the corresponding hidden tags to words. Spatial configurations and geometric characteristics of objects on the tabletop are described through 3D point cloud information that encodes spatial semantics and categories of referents and landmarks in the environment. The proposed model is evaluated through human user interaction with Toyota HSR robot, where the obtained results show the significant effect of the model in making the robot able to successfully engage in interaction with the user in space.

Research paper thumbnail of A Bayesian Approach to Phrase Understanding through Cross-Situational Learning

Proceedings of the International Workshop on Visually Grounded Interaction and Language (ViGIL), in Conjunction with the 32nd Conference on Neural Information Processing Systems (NeurIPS), Canada, 2018., 2018

In this paper, we present an unsupervised probabilistic framework to grounding words (e.g., nouns... more In this paper, we present an unsupervised probabilistic framework to grounding words (e.g., nouns, verbs, adjectives, and prepositions) through visual perception, and we discuss grammar induction in situated human-robot interaction with the objective of making a robot able to understand the underlying syntactic structure of human instructions so as to collaborate with users in space efficiently.

Research paper thumbnail of Editorial - Towards Intelligent Social Robots: Current Advances in Cognitive Robotics

Cognitive Systems Research (CSR) Journal, 2017

This special issue aims at shedding light on the intersection of cognitive science and robotics f... more This special issue aims at shedding light on the intersection of cognitive science and robotics from the theoretical and technical aspects, covering the basic research and its
applications. The recent advances and the future scope of cognitive robotics including the new methodologies, applied technologies, and robots are principal topics to be addressed in this special issue.

Research paper thumbnail of Social cognitive systems in smart environments: Approaches for learning, reasoning, and adaptation

Cognitive Systems Research, 2019

Robots with social competencies are no longer confinedto controlled laboratory environments as res... more Robots with social competencies are no longer confinedto controlled laboratory environments as research prototypes. They are beginning to appear in the real world as informational guides in airports, museums, and hospitals, collaborating closely alongside humans on factor floors, and assisting families in the home. As robots move into these new spaces, they must grapple with challenging and dynamic environments, requiring them to learn and adapt over time to unforeseen situations. The most important features of these environments are the humans that these robots need to work with and alongside, requiring them to​ learn and adapt specifically to these humans’ cognitive and behavioral characteristics.

Research paper thumbnail of Evaluation of Word Representations in Grounding Natural Language Instructions through Computational Human-Robot Interaction

Proceedings of the 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2019

Abstract—In order to interact with people in a natural way, a robot must be able to link words to... more Abstract—In order to interact with people in a natural way, a robot must be able to link words to objects and actions. Although previous studies in the literature have investigated grounding, they did not consider grounding of unknown synonyms. In this paper, we introduce a probabilistic model for grounding unknown synonymous object and action names using cross-situational learning. The proposed Bayesian learning model uses four different word representations to determine synonymous words. Afterwards, they are grounded through geometric characteristics of objects and kinematic features of the robot joints during action execution. The proposed model is evaluated through an interaction experiment between a human tutor and HSR robot. The results show that semantic and syntactic information both enable grounding of unknown synonyms and that the combination of both achieves the best grounding.

Research paper thumbnail of A Probabilistic Approach to Unsupervised Induction of Combinatory Categorial Grammar in Situated Human-Robot Interaction

IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids) Beijing, China, November 6-9, 2018, 2018

Abstract—Robots are progressively moving into spaces that have been primarily shaped by human age... more Abstract—Robots are progressively moving into spaces that have been primarily shaped by human agency; they collaborate with human users in different tasks that require them to understand human language so as to behave appropriately in space. To this end, a stubborn challenge that we address in this paper is inferring the syntactic structure of language,
which embraces grounding parts of speech (e.g., nouns, verbs, and prepositions) through visual perception, and induction of Combinatory Categorial Grammar (CCG) in situated human-robot interaction. This could pave the way towards making a robot able to understand the syntactic relationships between words (i.e., understand phrases), and consequently the meaning of human instructions during interaction, which is a future scope of this current study.

Research paper thumbnail of A Probabilistic Framework for Comparing Syntactic and Semantic Grounding of Synonyms through Cross-Situational Learning

Conference: ICRA-2018 Workshop on "Representing a Complex World: Perception, Inference, and Learning for Joint Semantic, Geometric, and Physical Understanding", Brisbane, Australia, 2018

Natural human-robot interaction requires robots to link words to objects and actions through grou... more Natural human-robot interaction requires robots to link words to objects and actions through grounding. Although grounding has been investigated in previous studies, none of them considered grounding of synonyms. In this paper, we try to fill this gap by introducing a Bayesian learning model for grounding synonymous object and action names using cross-situational learning. Three different word representations are employed with the probabilistic model and evaluated according to their grounding performance. Words are grounded through geometric characteristics of objects and kinematic features of the robot joints during action execution. An interaction experiment between a human tutor and HSR robot is used to evaluate the proposed model. The results show that representing words by syntactic and/or semantic information achieves worse grounding results than representing them by unique numbers.

Research paper thumbnail of Towards Understanding Object-Directed Actions: A Generative Model for Grounding Syntactic Categories of Speech through Visual Perception

Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Australia, 2018

Creating successful human-robot collaboration requires robots to have high-level cognitive functi... more Creating successful human-robot collaboration requires robots to have high-level cognitive functions that could allow them to understand human language and actions in space. To meet this target, an elusive challenge that we address in this paper is to understand object-directed actions through grounding language based on visual cues representing the dynamics of human actions on objects, object characteristics (color and geometry), and spatial relationships between objects in a tabletop scene. The proposed probabilistic framework investigates unsupervised Part-of-Speech (POS) tagging to determine syntactic categories of words so as to infer grammatical structure of language. The dynamics of object-directed actions are characterized through the locations of the human arm joints-modeled on a Hidden Markov Model (HMM)-while manipulating objects, in addition to those of objects represented in 3D point clouds. These corresponding point clouds to segmented objects encode geometric features and spatial semantics of referents and landmarks in the environment. The proposed Bayesian learning model is successfully evaluated through interaction experiments between a human user and Toyota HSR robot in space.

Research paper thumbnail of A Generative Framework for Multimodal Learning of Spatial Concepts and Object Categories: An Unsupervised Part-of-Speech Tagging and 3D Visual Perception Based Approach

Proceedings of the 7th Joint IEEE International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EPIROB), Portugal, 2017

Future human-robot collaboration employs language in instructing a robot about specific tasks to ... more Future human-robot collaboration employs language in instructing a robot about specific tasks to perform in its surroundings. This requires the robot to be able to associate spatial knowledge with language to understand the details of an assigned task so as to behave appropriately in the context of interaction. In this paper, we propose a probabilistic framework for learning the meaning of language spatial concepts (spatial prepositions) and object categories based on visual cues representing spatial layouts and geometric characteristics of objects in a tabletop scene. The model investigates unsupervised Part-of-Speech (POS) tagging through a Hidden Markov Model (HMM) that infers the corresponding hidden tags to words. Spatial configurations and geometric characteristics of objects on the tabletop are described through 3D point cloud information that encodes spatial semantics and categories of referents and landmarks in the environment. The proposed model is evaluated through human user interaction with Toyota HSR robot, where the obtained results show the significant effect of the model in making the robot able to successfully engage in interaction with the user in space.

Research paper thumbnail of A Bayesian Approach to Phrase Understanding through Cross-Situational Learning

Proceedings of the International Workshop on Visually Grounded Interaction and Language (ViGIL), in Conjunction with the 32nd Conference on Neural Information Processing Systems (NeurIPS), Canada, 2018., 2018

In this paper, we present an unsupervised probabilistic framework to grounding words (e.g., nouns... more In this paper, we present an unsupervised probabilistic framework to grounding words (e.g., nouns, verbs, adjectives, and prepositions) through visual perception, and we discuss grammar induction in situated human-robot interaction with the objective of making a robot able to understand the underlying syntactic structure of human instructions so as to collaborate with users in space efficiently.

Research paper thumbnail of Editorial - Towards Intelligent Social Robots: Current Advances in Cognitive Robotics

Cognitive Systems Research (CSR) Journal, 2017

This special issue aims at shedding light on the intersection of cognitive science and robotics f... more This special issue aims at shedding light on the intersection of cognitive science and robotics from the theoretical and technical aspects, covering the basic research and its
applications. The recent advances and the future scope of cognitive robotics including the new methodologies, applied technologies, and robots are principal topics to be addressed in this special issue.