Matthias Nieuwenhuisen | Rheinische Friedrich-Wilhelms-Universität Bonn (original) (raw)
Papers by Matthias Nieuwenhuisen
Journal of Intelligent & Robotic Systems, 2015
2015 European Conference on Mobile Robots (ECMR), 2015
2015 International Conference on Unmanned Aircraft Systems (ICUAS), 2015
Micro aerial vehicles, such as multirotors, are particular well suited for the autonomous monitor... more Micro aerial vehicles, such as multirotors, are particular well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance in industrial plants. Key prerequisites for the fully autonomous operation of micro aerial vehicles in complex 3D environments include real-time state estimation, obstacle detection, mapping, and navigation planning. In this paper, we describe an integrated system with a multimodal sensor setup for omnidirectional environment perception and 6D state estimation. Our MAV is equipped with a variety of sensors including a dual 3D laser scanner, three stereo camera pairs, an IMU and a powerful onboard computer to achieve these tasks in real-time. Our experimental evaluation demonstrates the performance of the integrated system.
Journal of Field Robotics, 2015
Micro aerial vehicles, such as multirotors, are particularly well suited for the autonomous monit... more Micro aerial vehicles, such as multirotors, are particularly well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance or disaster management. Key prerequisites for the fully autonomous operation of micro aerial vehicles are real-time obstacle detection and planning of collision-free trajectories. In this article, we propose a complete system with a multimodal sensor setup for omnidirectional obstacle perception consisting of a 3D laser scanner, two stereo camera pairs, and ultrasonic distance sensors. Detected obstacles are aggregated in egocentric local multiresolution grid maps. Local maps are efficiently merged in order to simultaneously build global maps of the environment and localize in these. For autonomous navigation, we generate trajectories in a multi-layered approach: from mission planning over global and local trajectory planning to reactive obstacle avoidance. We evaluate our approach and the involved components in simulation and with the real autonomous micro aerial vehicle. Finally, we present the results of a complete mission for autonomously mapping a building and its surroundings.
ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013
Limiting factors for increasing autonomy and complexity of truly autonomous systems (without exte... more Limiting factors for increasing autonomy and complexity of truly autonomous systems (without external sensing and control) are onboard sensing and onboard processing power. In this paper, we propose a hardware setup and processing pipeline that allows a fully autonomous UAV to perceive obstacles in (almost) all directions in its surroundings. Different sensor modalities are applied in order take into account the different characteristics of obstacles that can commonly be found in typical UAV applications. We provide a complete overview on the implemented system and present experimental results as a proof of concept.
ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013
International Journal of Social Robotics, 2013
The operation of robotic tour guides in public museums leads to a variety of interactions of thes... more The operation of robotic tour guides in public museums leads to a variety of interactions of these complex technical systems with humans of all ages and with different technical backgrounds. Interacting with a robot is a new experience for many visitors. An intuitive user interface, preferable one that resembles the interaction between human tour guides and visitors, simplifies the communication between robot and visitors. To allow for supportive behavior of the guided persons, predictable robot behavior is necessary. Humanoid robots are able to resemble human motions and behaviors and look familiar to human users that have not interacted with robots so far. Hence, they are particularly well suited for this purpose.
We present an approach to continuous motion planning with multiresolution in time. Our approach i... more We present an approach to continuous motion planning with
multiresolution in time. Our approach is based on stochastic trajectory
optimization for motion planning (STOMP) and designed to decrease the
optimization time in order to enable frequent replanning. Since service
robots operate in environments with dynamic obstacles, it is likely that
planned trajectories become invalid over time. Thus, it is not necessary to
provide trajectories with a uniform high resolution. Our multiresolutional
approach implicitly considers the uncertainty of the future by providing
a trajectory with a gradually coarser schedule, which is re ned trough
replanning. In addition to employing temporal multiresolution, we speed
up trajectory optimization by initializing replanning with the previous
plan. The proposed multiresolution STOMP is evaluated in simulation
in comparison to the original STOMP implementation. Our experiments
show that multiresolution STOMP reduces the planning time and, hence,
is able to avoid dynamic obstacles.
Micro aerial vehicles (MAVs), such as multicopters, are particular well suited for the inspection... more Micro aerial vehicles (MAVs), such as multicopters, are particular well suited for the inspection of human-built structures,
e. g., for maintenance or disaster management. Today, the operation of MAVs in the close vicinity of these structures
requires a human operator to remotely control the vehicle. For fully autonomous operation, a detailed model of the
environment is essential.
Building such a model by means of autonomous exploration is time consuming and delays the execution of the main
mission. In many real-world applications, a coarse model of the environment already exists and can be used for highlevel
planning. Nevertheless, detailed obstacle maps, needed for safe navigation, are often not available. We employ the
coarse information for global mission and path planning and refine the path on the fly, whenever the vehicle can acquire
information with its onboard sensors. To allow for fast replanning during the flight, we present a 3D local multiresolution
path planning approach making online grid-based planning for our MAV platform tractable.
For situations, where mapping is neither possible from high altitudes nor from the ground, we are... more For situations, where mapping is neither possible from high altitudes nor from the ground, we are developing an autonomous micro aerial vehicle able to fly at low altitudes in close vicinity of obstacles. This vehicle is based on a MikroKopter TM octocopter platform (maximum total weight: 5kg), and contains a dual frequency GPS board, an IMU, a compass, two stereo camera pairs with fisheye lenses, a rotating 3D laser scanner, 8 ultrasound sensors, a real-time processing unit, and a compact PC for on-board ego-motion estimation and obstacle detection for autonomous navigation. A high-resolution camera is used for the actual mapping task, where the environment is reconstructed in three dimensions from images, using a highly accurate bundle adjustment. In this contribution, we describe the sensor system setup and present results from the evaluation of several aspects of the different subsystems as well as initial results from flight tests.
Obstacle detection and real-time planning of collision-free trajectories are key for the fully au... more Obstacle detection and real-time planning of
collision-free trajectories are key for the fully autonomous
operation of micro aerial vehicles in restricted environments.
In this paper, we propose a complete system with a multimodal
sensor setup for omnidirectional obstacle perception
consisting of a 3D laser scanner, two stereo camera pairs, and
ultrasonic distance sensors. Detected obstacles are aggregated
in egocentric local multiresolution grid maps. We generate
trajectories in a multi-layered approach: from mission planning
to global and local trajectory planning, to reactive obstacle
avoidance.
We evaluate our approach in simulation and with the real
autonomous micro aerial vehicle.
Springer Tracts in Advanced Robotics, 2014
Grasping individual objects from an unordered pile in a box has been investigated in stationary s... more Grasping individual objects from an unordered pile in a box has been investigated in stationary scenarios so far. In this work, we present a complete system including active object perception and grasp planning for bin picking with a mobile robot. At the core of our approach is an efficient representation of objects as compounds of simple shape and contour primitives. This representation is used for both robust object perception and efficient grasp planning. For being able to manipulate previously unknown objects, we learn object models from single scans in an offline phase. During operation, objects are detected in the scene using a particularly robust probabilistic graph matching. To cope with severe occlusions we employ active perception considering not only previously unseen volume but also outcomes of primitive and object detection. The combination of shape and contour primitives makes our object perception approach particularly robust even in the presence of noise, occlusions, and missing information. For grasp planning, we efficiently pre-compute possible grasps directly on the learned object models. During operation, grasps and arm motions are planned in an efficient local multiresolution height map. All components are integrated and evaluated in a bin picking and part delivery task.
2013 IEEE International Conference on Robotics and Automation, 2013
Grasping individual objects from an unordered pile in a box has been investigated in static scena... more Grasping individual objects from an unordered pile in a box has been investigated in static scenarios so far. In this paper, we demonstrate bin picking with an anthropomorphic mobile robot. To this end, we extend global navigation techniques by precise local alignment with a transport box. Objects are detected in range images using a shape primitivebased approach. Our approach learns object models from single scans and employs active perception to cope with severe occlusions. Grasps and arm motions are planned in an efficient local multiresolution height map. All components are integrated and evaluated in a bin picking and part delivery task.
5th ACM/IEEE International Conference on Human-Robot Interaction, HRI 2010, 2010
Domestic service tasks require three main skills from autonomous robots: robust navigation in ind... more Domestic service tasks require three main skills from autonomous robots: robust navigation in indoor environments, flexible object manipulation, and intuitive communication with the users. In this report, we present the communication skills of our anthropomorphic service and communication robots Dynamaid and Robotinho. Both robots are equipped with an intuitive multimodal communication system, including speech synthesis and recognition, gestures and mimic. We evaluate our systems in the @Home league of the RoboCup competitions and in a museum tour guide scenario.
2010 10th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2010, 2010
Deploying robots at public places exposes highly complex systems to a variety of potential intera... more Deploying robots at public places exposes highly complex systems to a variety of potential interaction partners of all ages and with different technical backgrounds. Most of these individuals may have never interacted with a robot before. This raises the need for robots with an intuitive user interface, usable without prior training. Furthermore, predictable robot behavior is essential to allow for cooperative behavior on the human side. Humanoid robots are advantageous for this purpose, as they look familiar to persons without robotic experience. Moreover, they are able to resemble human motions and behaviors, allowing intuitive human-robot-interaction.
Journal of Intelligent & Robotic Systems, 2015
2015 European Conference on Mobile Robots (ECMR), 2015
2015 International Conference on Unmanned Aircraft Systems (ICUAS), 2015
Micro aerial vehicles, such as multirotors, are particular well suited for the autonomous monitor... more Micro aerial vehicles, such as multirotors, are particular well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance in industrial plants. Key prerequisites for the fully autonomous operation of micro aerial vehicles in complex 3D environments include real-time state estimation, obstacle detection, mapping, and navigation planning. In this paper, we describe an integrated system with a multimodal sensor setup for omnidirectional environment perception and 6D state estimation. Our MAV is equipped with a variety of sensors including a dual 3D laser scanner, three stereo camera pairs, an IMU and a powerful onboard computer to achieve these tasks in real-time. Our experimental evaluation demonstrates the performance of the integrated system.
Journal of Field Robotics, 2015
Micro aerial vehicles, such as multirotors, are particularly well suited for the autonomous monit... more Micro aerial vehicles, such as multirotors, are particularly well suited for the autonomous monitoring, inspection, and surveillance of buildings, e.g., for maintenance or disaster management. Key prerequisites for the fully autonomous operation of micro aerial vehicles are real-time obstacle detection and planning of collision-free trajectories. In this article, we propose a complete system with a multimodal sensor setup for omnidirectional obstacle perception consisting of a 3D laser scanner, two stereo camera pairs, and ultrasonic distance sensors. Detected obstacles are aggregated in egocentric local multiresolution grid maps. Local maps are efficiently merged in order to simultaneously build global maps of the environment and localize in these. For autonomous navigation, we generate trajectories in a multi-layered approach: from mission planning over global and local trajectory planning to reactive obstacle avoidance. We evaluate our approach and the involved components in simulation and with the real autonomous micro aerial vehicle. Finally, we present the results of a complete mission for autonomously mapping a building and its surroundings.
ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013
Limiting factors for increasing autonomy and complexity of truly autonomous systems (without exte... more Limiting factors for increasing autonomy and complexity of truly autonomous systems (without external sensing and control) are onboard sensing and onboard processing power. In this paper, we propose a hardware setup and processing pipeline that allows a fully autonomous UAV to perceive obstacles in (almost) all directions in its surroundings. Different sensor modalities are applied in order take into account the different characteristics of obstacles that can commonly be found in typical UAV applications. We provide a complete overview on the implemented system and present experimental results as a proof of concept.
ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013
International Journal of Social Robotics, 2013
The operation of robotic tour guides in public museums leads to a variety of interactions of thes... more The operation of robotic tour guides in public museums leads to a variety of interactions of these complex technical systems with humans of all ages and with different technical backgrounds. Interacting with a robot is a new experience for many visitors. An intuitive user interface, preferable one that resembles the interaction between human tour guides and visitors, simplifies the communication between robot and visitors. To allow for supportive behavior of the guided persons, predictable robot behavior is necessary. Humanoid robots are able to resemble human motions and behaviors and look familiar to human users that have not interacted with robots so far. Hence, they are particularly well suited for this purpose.
We present an approach to continuous motion planning with multiresolution in time. Our approach i... more We present an approach to continuous motion planning with
multiresolution in time. Our approach is based on stochastic trajectory
optimization for motion planning (STOMP) and designed to decrease the
optimization time in order to enable frequent replanning. Since service
robots operate in environments with dynamic obstacles, it is likely that
planned trajectories become invalid over time. Thus, it is not necessary to
provide trajectories with a uniform high resolution. Our multiresolutional
approach implicitly considers the uncertainty of the future by providing
a trajectory with a gradually coarser schedule, which is re ned trough
replanning. In addition to employing temporal multiresolution, we speed
up trajectory optimization by initializing replanning with the previous
plan. The proposed multiresolution STOMP is evaluated in simulation
in comparison to the original STOMP implementation. Our experiments
show that multiresolution STOMP reduces the planning time and, hence,
is able to avoid dynamic obstacles.
Micro aerial vehicles (MAVs), such as multicopters, are particular well suited for the inspection... more Micro aerial vehicles (MAVs), such as multicopters, are particular well suited for the inspection of human-built structures,
e. g., for maintenance or disaster management. Today, the operation of MAVs in the close vicinity of these structures
requires a human operator to remotely control the vehicle. For fully autonomous operation, a detailed model of the
environment is essential.
Building such a model by means of autonomous exploration is time consuming and delays the execution of the main
mission. In many real-world applications, a coarse model of the environment already exists and can be used for highlevel
planning. Nevertheless, detailed obstacle maps, needed for safe navigation, are often not available. We employ the
coarse information for global mission and path planning and refine the path on the fly, whenever the vehicle can acquire
information with its onboard sensors. To allow for fast replanning during the flight, we present a 3D local multiresolution
path planning approach making online grid-based planning for our MAV platform tractable.
For situations, where mapping is neither possible from high altitudes nor from the ground, we are... more For situations, where mapping is neither possible from high altitudes nor from the ground, we are developing an autonomous micro aerial vehicle able to fly at low altitudes in close vicinity of obstacles. This vehicle is based on a MikroKopter TM octocopter platform (maximum total weight: 5kg), and contains a dual frequency GPS board, an IMU, a compass, two stereo camera pairs with fisheye lenses, a rotating 3D laser scanner, 8 ultrasound sensors, a real-time processing unit, and a compact PC for on-board ego-motion estimation and obstacle detection for autonomous navigation. A high-resolution camera is used for the actual mapping task, where the environment is reconstructed in three dimensions from images, using a highly accurate bundle adjustment. In this contribution, we describe the sensor system setup and present results from the evaluation of several aspects of the different subsystems as well as initial results from flight tests.
Obstacle detection and real-time planning of collision-free trajectories are key for the fully au... more Obstacle detection and real-time planning of
collision-free trajectories are key for the fully autonomous
operation of micro aerial vehicles in restricted environments.
In this paper, we propose a complete system with a multimodal
sensor setup for omnidirectional obstacle perception
consisting of a 3D laser scanner, two stereo camera pairs, and
ultrasonic distance sensors. Detected obstacles are aggregated
in egocentric local multiresolution grid maps. We generate
trajectories in a multi-layered approach: from mission planning
to global and local trajectory planning, to reactive obstacle
avoidance.
We evaluate our approach in simulation and with the real
autonomous micro aerial vehicle.
Springer Tracts in Advanced Robotics, 2014
Grasping individual objects from an unordered pile in a box has been investigated in stationary s... more Grasping individual objects from an unordered pile in a box has been investigated in stationary scenarios so far. In this work, we present a complete system including active object perception and grasp planning for bin picking with a mobile robot. At the core of our approach is an efficient representation of objects as compounds of simple shape and contour primitives. This representation is used for both robust object perception and efficient grasp planning. For being able to manipulate previously unknown objects, we learn object models from single scans in an offline phase. During operation, objects are detected in the scene using a particularly robust probabilistic graph matching. To cope with severe occlusions we employ active perception considering not only previously unseen volume but also outcomes of primitive and object detection. The combination of shape and contour primitives makes our object perception approach particularly robust even in the presence of noise, occlusions, and missing information. For grasp planning, we efficiently pre-compute possible grasps directly on the learned object models. During operation, grasps and arm motions are planned in an efficient local multiresolution height map. All components are integrated and evaluated in a bin picking and part delivery task.
2013 IEEE International Conference on Robotics and Automation, 2013
Grasping individual objects from an unordered pile in a box has been investigated in static scena... more Grasping individual objects from an unordered pile in a box has been investigated in static scenarios so far. In this paper, we demonstrate bin picking with an anthropomorphic mobile robot. To this end, we extend global navigation techniques by precise local alignment with a transport box. Objects are detected in range images using a shape primitivebased approach. Our approach learns object models from single scans and employs active perception to cope with severe occlusions. Grasps and arm motions are planned in an efficient local multiresolution height map. All components are integrated and evaluated in a bin picking and part delivery task.
5th ACM/IEEE International Conference on Human-Robot Interaction, HRI 2010, 2010
Domestic service tasks require three main skills from autonomous robots: robust navigation in ind... more Domestic service tasks require three main skills from autonomous robots: robust navigation in indoor environments, flexible object manipulation, and intuitive communication with the users. In this report, we present the communication skills of our anthropomorphic service and communication robots Dynamaid and Robotinho. Both robots are equipped with an intuitive multimodal communication system, including speech synthesis and recognition, gestures and mimic. We evaluate our systems in the @Home league of the RoboCup competitions and in a museum tour guide scenario.
2010 10th IEEE-RAS International Conference on Humanoid Robots, Humanoids 2010, 2010
Deploying robots at public places exposes highly complex systems to a variety of potential intera... more Deploying robots at public places exposes highly complex systems to a variety of potential interaction partners of all ages and with different technical backgrounds. Most of these individuals may have never interacted with a robot before. This raises the need for robots with an intuitive user interface, usable without prior training. Furthermore, predictable robot behavior is essential to allow for cooperative behavior on the human side. Humanoid robots are advantageous for this purpose, as they look familiar to persons without robotic experience. Moreover, they are able to resemble human motions and behaviors, allowing intuitive human-robot-interaction.