Nikhil Somani | Technische Universität München (original) (raw)

Papers by Nikhil Somani

CAD/CAM approaches have been used in the manufacturing industry for a long time, and their use in... more CAD/CAM approaches have been used in the manufacturing industry for a long time, and their use in robotic systems is becoming more popular. One common element in these approaches is the use of geometric constraints to define relative object poses. Hence, approaches for solving these geometric constraints are critical to their performance. In this work, we present an exact solver for geometric constraints. Our approach is based on mathematical models of constraints and geometric properties of constraint nullspaces. Our constraint solver supports non-linear constraints with inequalities, and also mixed transformation manifolds, i.e., cases where the rotation and translation components of the constraints are not independent. Through several applications, we show how inequality constraints and mixed transformation manifolds increase the expressive power of constraint-based task definitions. The exact solver provides repeatable solutions with deterministic runtimes and our experiments show that it is also much faster than comparable iterative solvers.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

In this paper, we propose a framework for prioritized constraint-based specification of robot tas... more In this paper, we propose a framework for prioritized constraint-based specification of robot tasks. This framework is integrated with a cognitive robotic system based on semantic models of processes, objects, and workcells. The target is to enable intuitive (reprogramming g of robot tasks, in a way that is suitable for non-expert users typically found in SMEs. Using CAD semantics, robot tasks are specified as geometric inter-relational constraints. During execution, these are combined with constraints from the environment and the workcell, and solved in real-time. Our constraint model and solving approach supports a variety of constraint functions that can be non-linear and also include bounds in the form of inequalities, e.g., geometric interrelations , distance, collision avoidance and posture constraints. It is a hierarchical approach where priority levels can be specified for the constraints, and the nullspace of higher priority constraints is exploited to optimize the lower priority constraints. The presented approach has been applied to several typical industrial robotic use-cases to highlight its advantages compared to other state-of-the-art approaches.

This paper details and discusses work performed at the woodworking SME Mivelaz Techniques Bois SA... more This paper details and discusses work performed at the woodworking SME Mivelaz Techniques Bois SA within the SME-robotics FP7 project. The aim is to improve non-expert handling of the cell by introduction of cognitive abilities in the robot system. Three areas are considered; intuitive programming, process adaptation and system integration. Proposed cognitive components are described together with experiments performed.

We present an approach for monitoring and interpreting human activities based on a novel multimod... more We present an approach for monitoring and interpreting human activities based on a novel multimodal vision-based interface, aiming at improving the efficiency of human-robot interaction (HRI) in industrial environments.

Proceedings of the Workshop on Multimodal Semantics for Robotic Systems, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep 2015

In this paper, we describe our ongoing efforts to design a cognition-enabled industrial robotic w... more In this paper, we describe our ongoing efforts to design a cognition-enabled industrial robotic workcell, which significantly increases the efficiency of teaching and adapting robot tasks. We have designed a formalism to match task parameter and input modality types, in order to infer suitable means for binding values to those parameters. All modalities are integrated through a graphical user interface, which a human operator can use to program industrial robots in an intuitive way by arbitrarily choosing modalities according to his or her preference.

Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), Dec 2015

To synthesize whole-body behaviors interactively, multiple tasks and constraints need to be simul... more To synthesize whole-body behaviors interactively, multiple tasks and constraints need to be simultaneously controlled, including those that guarantee that the constraints imposed by the robot's structure and the external environment are satisfied. In this paper, we present a prioritized, multiple-task control framework that is able to control forces in systems ranging from humanoids to industrial robots. Priorities between tasks are accomplished through null-space projection. Several relevant constraints (i.e., motion constraints, joint limits, force control) are tested to evaluate the control framework. Further, we evaluate the proposed approach in two typical industrial robotics applications: grasping of cylindrical objects and welding.

Proceedings of the IEEE International Conference on Robotics and Biomimetics, Dec 2015

In this paper, an approach for matching of primitive shapes detected from point clouds, to bounda... more In this paper, an approach for matching of primitive shapes detected from point clouds, to boundary representations of primitive shapes contained in CAD models of objects/workpieces is presented. The primary target application is object detection and pose estimation from noisy RGBD sensor data. This approach can also be used to determine incomplete object poses, including those of symmetrical objects. Detection and reasoning about these under-specified object poses is useful in several practical applications such as robotic manipulation, which are also presented in this paper.

Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep 2015

In this paper, we introduce an approach for leveraging CAD description to a semantic level, in or... more In this paper, we introduce an approach for leveraging CAD description to a semantic level, in order to link additional knowledge to CAD models and to exploit resulting synergy effects. This has been achieved by designing a description language, based on the Web Ontology Language (OWL), that is used to define boundary representations (BREP) of objects. This involves representing geometric entities in a semantic meaningful way, e.g., a circle is defined by a coordinate frame and a radius instead of a set of polygons. Furthermore, the scope of this semantic description language also covers geometric constraints between multiple objects. Constraints can be specified not only on the object level, but down to single edges or faces of an object. This semantic representation is used to improve a variety of applications, ranging from shape-based object recognition to constraint-based robot task descriptions. Results from a quantitative evaluation are presented to assess the practicability of this approach.

Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep 2015

In this paper, we propose a framework for intuitive task-based programming of robots using geomet... more In this paper, we propose a framework for intuitive task-based programming of robots using geometric inter-relational constraints. The intended applications of this framework are robot programming interfaces that use semantically rich task descriptions, allow intuitive (reprogramming g, and are suitable for non-expert users typically found in SMEs. A key concept in this work is the use of CAD semantics to represent geometric entities in the robotic workcell. The robot tasks are then represented as a set of geometrical inter-relational constraints, which are solved in real-time to be executed on the robot. Since these constraints often specify the target pose only partially, the robot can be controlled to move in the constraints' null space in order to handle external disturbances or further optimize the robot's pose during runtime. Geometrical inter-relational constraints are easy to understand and can be intuitively specified using CAD software. A number of applications common in industrial robotic scenarios have been chosen to highlight the advantages of the presented approach vis-à-vis the state-of-the-art approaches.

Proceedings of the International Symposium on Visual Computing, Dec 2014

In this paper, an object recognition and pose estimation approach based on constraints from primi... more In this paper, an object recognition and pose estimation approach based on constraints from primitive shape matching is presented. Additionally, an approach for primitive shape detection from point clouds using an energy minimization formulation is presented. Each primitive shape in an object adds geometric constraints on the object's pose. An algorithm is proposed to find minimal sets of primitive shapes which are sufficient to determine the complete 3D position and orientation of a rigid object. The pose is estimated using a linear least squares solver over the combination of constraints enforced by the primitive shapes. Experiments illustrating the primitive shape decomposition of object models, detection of these minimal sets, feature vector calculation for sets of shapes and object pose estimation have been presented on simulated and real data.

Proceedings of the AsiaPacific Signal and Information Processing Association Annual Summit and Conference, Dec 2014

Human activity recognition is crucial for intuitive cooperation between humans and robots. We pre... more Human activity recognition is crucial for intuitive cooperation between humans and robots. We present an approach for activity recognition for applications in the context of human-robot interaction in industrial settings. The approach is based on spatial and temporal features derived from skeletal data of human workers performing assembly tasks. These features were used to train a machine learning framework, which classifies discrete time frames with Random Forests and subsequently models temporal dependencies between the resulting states with a Hidden Markov Model. We considered the following three groups of activities: Movement, Gestures, and Object handling. A dataset has been collected which is comprised of 24 recordings of several human workers performing such activities in a human-robot interaction environment, as typically seen at small and medium-sized enterprises. The evaluation shows that the approach achieves a recognition accuracy of up to 88% for some activities and an average accuracy of 73%.

Proceedings of the Workshop Towards Intelligent Social Robots - Current Advances in Cognitive Robotics, IEEE/RAS International Conference on Humanoid Robots, Nov 2015

In this paper, we present an integrated approach to knowledge representation for cognitive robots... more In this paper, we present an integrated approach to knowledge representation for cognitive robots. We combine knowledge about robot tasks, interaction objects including their geometric shapes, the environment, and natural language in a common ontological description. This description is based on the Web Ontology Language (OWL) and allows to automatically link and interpret these different kinds of information. Semantic descriptions are shared between object detection and pose estimation, task-level manipulation skills, and human-friendly interfaces. Through lifting the level of communication between the human operator and the robot system to an abstract level, we achieve more human-suitable interaction and thus a higher level of acceptance by the user. Furthermore, it increases the efficiency of communication. The benefits of our approach are highlighted by examples from the domains of industrial assembly and service robotics.

Intuitive programming of industrial robots is especially important for small and medium-sized ent... more Intuitive programming of industrial robots is especially important for small and medium-sized enterprises. We evaluated four different input modalities (touch, gesture, speech, 3D tracking device) regarding their preference, usability, and intuitiveness for robot programming.

A Wizard-of-Oz experiment was conducted with 30 participants and its results show that most users prefer touch and gesture input over 3D tracking device input, whereas speech input was the least preferred input modality. The results also indicate that there are gender specific differences for preferred input modalities.

We show how the results of the user study can be formalized in a semantic description language in such a way that a cognitive robotic workcell can benefit from the additional knowledge of input and output modalities, task parameter types, and preferred combinations of the two.

2014 13th International Conference on Control Automation Robotics & Vision (ICARCV), 2014

Lecture Notes in Computer Science, 2013

2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014