Pablo Gil | University of Alicante / Universidad de Alicante (original) (raw)

Papers by Pablo Gil

Journal of Intelligent & Robotic Systems

The accumulation of litter is increasing in many places and is consequently becoming a problem th... more The accumulation of litter is increasing in many places and is consequently becoming a problem that must be dealt with. In this paper, we present a manipulator robotic system to collect litter in outdoor environments. This system has three functionalities. Firstly, it uses colour images to detect and recognise litter comprising different materials. Secondly, depth data are combined with pixels of waste objects to compute a 3D location and segment three-dimensional point clouds of the litter items in the scene. The grasp in 3 Degrees of Freedom (DoFs) is then estimated for a robot arm with a gripper for the segmented cloud of each instance of waste. Finally, two tactile-based algorithms are implemented and then employed in order to provide the gripper with a sense of touch. This work uses two low-cost visual-based tactile sensors at the fingertips. One of them addresses the detection of contact (which is obtained from tactile images) between the gripper and solid waste, while another...

Actas de las XXXVII Jornadas de Automática 7, 8 y 9 de septiembre de 2016, Madrid, 2022

En este artículo se presentan dos métodos de segmentación para la detección de vertidos de hidroc... more En este artículo se presentan dos métodos de segmentación para la detección de vertidos de hidrocarburos en la superficie marítima a partir de imágenes obtenidas por un sensor SLAR embarcado en una aeronave. Para ello, se describen y comparan dos aproximaciones de segmentación, basadas en grafo e imagen-J, respectivamente. Finalmente, se muestra el resultado de aplicar ambas aproximaciones a imágenes SLAR, buscando como objetivo detectar la mayor área de vertido en la superficie marina al tiempo que se minimiza la falsa detección de ésta.

XLV Jornadas de Automática, 2024

Optimal robotic grasping cannot be limited to the estimation of object grasping pose using vision... more Optimal robotic grasping cannot be limited to the estimation of object grasping pose using vision-based methods. It is necessary to use tactile sensors to learn the physical properties of the objects that are to be grasped. In this work, we integrated two Contactile force-based tactile sensors with a 2F-140 ROBOTIQ gripper and a UR5 robot to estimate the volume of a waterfilled container using Multilayer Perceptron (MLP) neural networks. During experimentation, we trained and evaluated different
MLPs varying the input forces (Fx, Fy, Fz) in a task of discrete-volume regression in a range of between 0ml and 300ml. The preliminary proposed approach is compared with an algebraic method based on the diagram of the equilibrium of forces, proving that our results are more precise, obtaining a R2 value of 8% higher in the worst-case scenario, and of 30% in the best.

The International Journal of Advanced Manufacturing Technology

The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked ... more The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked on pallets to be supplied to its customers. Human operators usually classify these pallets according to the physical features of the cardboard packaging. This process can be slow, causing congestion on the production line. To optimise the logistics of this process, we propose a visual recognition and tracking pipeline that monitors the palletised packaging while it is moving inside the factory on roller conveyors. Our pipeline has a two-stage architecture composed of Convolutional Neural Networks, one for oriented pallet detection and recognition, and another with which to track identified pallets. We carried out an extensive study using different methods for the pallet detection and tracking tasks and discovered that the oriented object detection approach was the most suitable. Our proposal recognises and tracks different configurations and visual appearance of palletised packaging, pro...

Our solution for learning to regress tactile responses from visual perception is composed of the ... more Our solution for learning to regress tactile responses from visual perception is composed of the following: Robotic System: Shadow dexterous hand equipped with BioTac SP sensors and mounted on a Mitsubishi PA10. Vision is acquired from a Intel RealSense D415 depth camera. We only use middle finger and thumb for grasps. Visual Representation: We use 3D point clouds as a visual stimulus for this learning task. Objects are segmented from the background and only position information is used. That is, points only hold 3D coordinates. Network Architecture: Modified PointNet network for regression. Point clouds are downsampled to 500 points and normalised to the unit sphere with centre at the cloud's centroid. Tactile data are scaled to range [0, 1].

IEEE Robotics and Automation Letters, 2023

When carrying out robotic manipulation tasks, objects occasionally fall as a result of the rotati... more When carrying out robotic manipulation tasks, objects occasionally fall as a result of the rotation caused by slippage. This can be prevented by obtaining tactile information that provides better knowledge on the physical properties of the grasping. In this letter, we estimate the rotation angle of a grasped object when slippage occurs. We implement a system made up of a neural network with which to segment the contact region and an algorithm with which to estimate the rotated angle of that region. This method is applied to DIGIT tactile sensors. Our system has additionally been trained and tested with our publicly available dataset which is, to the best of our knowledge, the first dataset related to tactile segmentation from non-synthetic images to appear in the literature, and with which we have attained results of 95% and 90% as regards Dice and IoU metrics in the worst scenario. Moreover, we have obtained a maximum error of ≈3 degrees when testing with objects not previously seen by our system in 45 different lifts. This, therefore, proved that our approach is able to detect the slippage movement, thus providing a possible reaction that will prevent the object from falling.

IEEE Access, 2019

This paper presents a method that can be used for the efficient detection of small maritime objec... more This paper presents a method that can be used for the efficient detection of small maritime objects. The proposed method employs aerial images in the visible spectrum as inputs to train a categorical convolutional neural network for the classification of ships. A subset of those filters that make the greatest contribution to the classification of the target class is selected from the inner layers of the CNN. The gradients with respect to the input image are then calculated on these filters, which are subsequently normalized and combined. Thresholding and a morphological operation are then applied in order to eventually obtain the localization. One of the advantages of the proposed approach with regard to previous object detection methods is that it is only required to label a few images with bounding boxes of the targets to be trained for localization. The method was evaluated with an extended version of the MASATI (MAritime SATellite Imagery) dataset. This new dataset has more than 7 000 images, 4 157 of which contain ships. Using only 14 training images, the proposed approach achieves better results for small targets than other well-known object detection methods, which also require many more training images. INDEX TERMS Artificial neural networks, learning systems, object detection, remote sensing.

The International Journal of Advanced Manufacturing Technology, 2023

The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked ... more The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked on pallets to be supplied to its customers. Human operators usually classify these pallets according to the physical features of the cardboard packaging. This process can be slow, causing congestion on the production line. To optimise the logistics of this process, we propose a visual recognition and tracking pipeline that monitors the palletised packaging while it is moving inside the factory on roller conveyors. Our pipeline has a two-stage architecture composed of Convolutional Neural Networks, one for oriented pallet detection and recognition, and another with which to track identified pallets. We carried out an extensive study using different methods for the pallet detection and tracking tasks and discovered that the oriented object detection approach was the most suitable. Our proposal recognises and tracks different configurations and visual appearance of palletised packaging, providing statistical data in real time with which to assist human operators in decision-making. We tested the precision-performance of the system at the Smurfit Kappa facilities. Our proposal attained an Average Precision (AP) of 0.93 at 14 Frames Per Second (FPS), losing only 1% of detections. Our system is, therefore, able to optimise and speed up the process of logistic distribution.

Journal of Intelligent & Robotic Systems, 2023

The accumulation of litter is increasing in many places and is consequently becoming a problem th... more The accumulation of litter is increasing in many places and is consequently becoming a problem that must be dealt with. In this paper, we present a manipulator robotic system to collect litter in outdoor environments. This system has three functionalities. Firstly, it uses colour images to detect and recognise litter comprising different materials. Secondly, depth data are combined with pixels of waste objects to compute a 3D location and segment three-dimensional point clouds of the litter items in the scene. The grasp in 3 Degrees of Freedom (DoFs) is then estimated for a robot arm with a gripper for the segmented cloud of each instance of waste. Finally, two tactile-based algorithms are implemented and then employed in order to provide the gripper with a sense of touch. This work uses two low-cost visual-based tactile sensors at the fingertips. One of them addresses the detection of contact (which is obtained from tactile images) between the gripper and solid waste, while another has been designed to detect slippage in order to prevent the objects grasped from falling. Our proposal was successfully tested by carrying out extensive experimentation with different objects varying in size, texture, geometry and materials in different outdoor environments (a tiled pavement, a surface of stone/soil, and grass). Our system achieved an average score of 94% for the detection and Collection Success Rate (CSR) as regards its overall performance, and of 80% for the collection of items of litter at the first attempt.

Proceedings of the 5th International Symposium on Sensor Science (I3S 2017), 2017

Education in the Knowledge Society, 2012

This paper discusses different teaching experiences which aims are the learning robotics in the u... more This paper discusses different teaching experiences which aims are the learning robotics in the university. These experiences are reflected in the development of several robotics courses and subjects at the University of Alicante. The authors have created various educational platforms or they have used tools of free distribution and open source for the implementation of these courses. The main objetive of these courses is to teach the design and implementation of robotic solutions to solve various problems not only such as the control, programming and handling of robot but also the assembly, building and programming of educational mini-robots. On the one hand, new teaching tools are used such as simulators and virtual labs which make flexible the learning of robot arms. On the other hand, competitions are used to motivate students because this way, the students put into action the skills learned through building and programming low-cost mini-robots.

Latest trends in robotic grasping combine vision and touch for improving the performance of syste... more Latest trends in robotic grasping combine vision and touch for improving the performance of systems at tasks like stability prediction. However, tactile data are only available during the grasp, limiting the set of scenarios in which multimodal solutions can be applied. Could we obtain it prior to grasping? We explore the use of visual perception as a stimulus for generating tactile data so the robotic system can ”feel” the response of the tactile perception just by looking at the object.

Sensors (Basel, Switzerland), Jan 20, 2018

This paper presents a system that combines computer vision and surface electromyography technique... more This paper presents a system that combines computer vision and surface electromyography techniques to perform grasping tasks with a robotic hand. In order to achieve a reliable grasping action, the vision-driven system is used to compute pre-grasping poses of the robotic system based on the analysis of tridimensional object features. Then, the human operator can correct the pre-grasping pose of the robot using surface electromyographic signals from the forearm during wrist flexion and extension. Weak wrist flexions and extensions allow a fine adjustment of the robotic system to grasp the object and finally, when the operator considers that the grasping position is optimal, a strong flexion is performed to initiate the grasping of the object. The system has been tested with several subjects to check its performance showing a grasping accuracy of around 95% of the attempted grasps which increases in more than a 13% the grasping accuracy of previous experiments in which electromyograph...

IEEE Transactions on Geoscience and Remote Sensing, 2018

This paper presents a system for the detection of ships and oil spills using side-looking airborn... more This paper presents a system for the detection of ships and oil spills using side-looking airborne radar (SLAR) images. The proposed method employs a two-stage architecture composed of three pairs of convolutional neural networks (CNNs). Each pair of networks is trained to recognize a single class (ship, oil spill, and coast) by following two steps: a first network performs a coarse detection, and then, a second specialized CNN obtains the precise localization of the pixels belonging to each class. After classification, a postprocessing stage is performed by applying a morphological opening filter in order to eliminate small look-alikes, and removing those oil spills and ships that are surrounded by a minimum amount of coast. Data augmentation is performed to increase the number of samples, owing to the difficulty involved in obtaining a sufficient number of correctly labeled SLAR images. The proposed method is evaluated and compared to a single multiclass CNN architecture and to previous state-of-the-art methods using accuracy, precision, recall, F-measure, and intersection over union. The results show that the proposed method is efficient and competitive, and outperforms the approaches previously used for this task.

Learning and teaching processes are continually changing. Therefore, design of learning technolog... more Learning and teaching processes are continually changing. Therefore, design of learning technologies has gained interest in educators and educational institutions from secondary school to higher education. This paper describes the successfully use in education of social learning technologies and virtual laboratories designed by the authors, as well as videos developed by the students. These tools, combined with other open educational resources based on a blended-learning methodology, have been employed to teach the subject of Computer Networks. We have verified not only that the application of OERs into the learning process leads to a significantly improvement of the assessments, but also that the combination of several OERs enhances their effectiveness. These results are supported by, firstly, a study of both students ’ opinion and students ’ behaviour over five academic years, and, secondly, a correlation analysis between the use of OERs and the grades obtained by students.

Journal of Intelligent & Robotic Systems

The accumulation of litter is increasing in many places and is consequently becoming a problem th... more The accumulation of litter is increasing in many places and is consequently becoming a problem that must be dealt with. In this paper, we present a manipulator robotic system to collect litter in outdoor environments. This system has three functionalities. Firstly, it uses colour images to detect and recognise litter comprising different materials. Secondly, depth data are combined with pixels of waste objects to compute a 3D location and segment three-dimensional point clouds of the litter items in the scene. The grasp in 3 Degrees of Freedom (DoFs) is then estimated for a robot arm with a gripper for the segmented cloud of each instance of waste. Finally, two tactile-based algorithms are implemented and then employed in order to provide the gripper with a sense of touch. This work uses two low-cost visual-based tactile sensors at the fingertips. One of them addresses the detection of contact (which is obtained from tactile images) between the gripper and solid waste, while another...

Actas de las XXXVII Jornadas de Automática 7, 8 y 9 de septiembre de 2016, Madrid, 2022

En este artículo se presentan dos métodos de segmentación para la detección de vertidos de hidroc... more En este artículo se presentan dos métodos de segmentación para la detección de vertidos de hidrocarburos en la superficie marítima a partir de imágenes obtenidas por un sensor SLAR embarcado en una aeronave. Para ello, se describen y comparan dos aproximaciones de segmentación, basadas en grafo e imagen-J, respectivamente. Finalmente, se muestra el resultado de aplicar ambas aproximaciones a imágenes SLAR, buscando como objetivo detectar la mayor área de vertido en la superficie marina al tiempo que se minimiza la falsa detección de ésta.

XLV Jornadas de Automática, 2024

Optimal robotic grasping cannot be limited to the estimation of object grasping pose using vision... more Optimal robotic grasping cannot be limited to the estimation of object grasping pose using vision-based methods. It is necessary to use tactile sensors to learn the physical properties of the objects that are to be grasped. In this work, we integrated two Contactile force-based tactile sensors with a 2F-140 ROBOTIQ gripper and a UR5 robot to estimate the volume of a waterfilled container using Multilayer Perceptron (MLP) neural networks. During experimentation, we trained and evaluated different
MLPs varying the input forces (Fx, Fy, Fz) in a task of discrete-volume regression in a range of between 0ml and 300ml. The preliminary proposed approach is compared with an algebraic method based on the diagram of the equilibrium of forces, proving that our results are more precise, obtaining a R2 value of 8% higher in the worst-case scenario, and of 30% in the best.

The International Journal of Advanced Manufacturing Technology

The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked ... more The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked on pallets to be supplied to its customers. Human operators usually classify these pallets according to the physical features of the cardboard packaging. This process can be slow, causing congestion on the production line. To optimise the logistics of this process, we propose a visual recognition and tracking pipeline that monitors the palletised packaging while it is moving inside the factory on roller conveyors. Our pipeline has a two-stage architecture composed of Convolutional Neural Networks, one for oriented pallet detection and recognition, and another with which to track identified pallets. We carried out an extensive study using different methods for the pallet detection and tracking tasks and discovered that the oriented object detection approach was the most suitable. Our proposal recognises and tracks different configurations and visual appearance of palletised packaging, pro...

Our solution for learning to regress tactile responses from visual perception is composed of the ... more Our solution for learning to regress tactile responses from visual perception is composed of the following: Robotic System: Shadow dexterous hand equipped with BioTac SP sensors and mounted on a Mitsubishi PA10. Vision is acquired from a Intel RealSense D415 depth camera. We only use middle finger and thumb for grasps. Visual Representation: We use 3D point clouds as a visual stimulus for this learning task. Objects are segmented from the background and only position information is used. That is, points only hold 3D coordinates. Network Architecture: Modified PointNet network for regression. Point clouds are downsampled to 500 points and normalised to the unit sphere with centre at the cloud's centroid. Tactile data are scaled to range [0, 1].

IEEE Robotics and Automation Letters, 2023

When carrying out robotic manipulation tasks, objects occasionally fall as a result of the rotati... more When carrying out robotic manipulation tasks, objects occasionally fall as a result of the rotation caused by slippage. This can be prevented by obtaining tactile information that provides better knowledge on the physical properties of the grasping. In this letter, we estimate the rotation angle of a grasped object when slippage occurs. We implement a system made up of a neural network with which to segment the contact region and an algorithm with which to estimate the rotated angle of that region. This method is applied to DIGIT tactile sensors. Our system has additionally been trained and tested with our publicly available dataset which is, to the best of our knowledge, the first dataset related to tactile segmentation from non-synthetic images to appear in the literature, and with which we have attained results of 95% and 90% as regards Dice and IoU metrics in the worst scenario. Moreover, we have obtained a maximum error of ≈3 degrees when testing with objects not previously seen by our system in 45 different lifts. This, therefore, proved that our approach is able to detect the slippage movement, thus providing a possible reaction that will prevent the object from falling.

IEEE Access, 2019

This paper presents a method that can be used for the efficient detection of small maritime objec... more This paper presents a method that can be used for the efficient detection of small maritime objects. The proposed method employs aerial images in the visible spectrum as inputs to train a categorical convolutional neural network for the classification of ships. A subset of those filters that make the greatest contribution to the classification of the target class is selected from the inner layers of the CNN. The gradients with respect to the input image are then calculated on these filters, which are subsequently normalized and combined. Thresholding and a morphological operation are then applied in order to eventually obtain the localization. One of the advantages of the proposed approach with regard to previous object detection methods is that it is only required to label a few images with bounding boxes of the targets to be trained for localization. The method was evaluated with an extended version of the MASATI (MAritime SATellite Imagery) dataset. This new dataset has more than 7 000 images, 4 157 of which contain ships. Using only 14 training images, the proposed approach achieves better results for small targets than other well-known object detection methods, which also require many more training images. INDEX TERMS Artificial neural networks, learning systems, object detection, remote sensing.

The International Journal of Advanced Manufacturing Technology, 2023

The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked ... more The paper industry manufactures corrugated cardboard packaging, which is unassembled and stacked on pallets to be supplied to its customers. Human operators usually classify these pallets according to the physical features of the cardboard packaging. This process can be slow, causing congestion on the production line. To optimise the logistics of this process, we propose a visual recognition and tracking pipeline that monitors the palletised packaging while it is moving inside the factory on roller conveyors. Our pipeline has a two-stage architecture composed of Convolutional Neural Networks, one for oriented pallet detection and recognition, and another with which to track identified pallets. We carried out an extensive study using different methods for the pallet detection and tracking tasks and discovered that the oriented object detection approach was the most suitable. Our proposal recognises and tracks different configurations and visual appearance of palletised packaging, providing statistical data in real time with which to assist human operators in decision-making. We tested the precision-performance of the system at the Smurfit Kappa facilities. Our proposal attained an Average Precision (AP) of 0.93 at 14 Frames Per Second (FPS), losing only 1% of detections. Our system is, therefore, able to optimise and speed up the process of logistic distribution.

Journal of Intelligent & Robotic Systems, 2023

The accumulation of litter is increasing in many places and is consequently becoming a problem th... more The accumulation of litter is increasing in many places and is consequently becoming a problem that must be dealt with. In this paper, we present a manipulator robotic system to collect litter in outdoor environments. This system has three functionalities. Firstly, it uses colour images to detect and recognise litter comprising different materials. Secondly, depth data are combined with pixels of waste objects to compute a 3D location and segment three-dimensional point clouds of the litter items in the scene. The grasp in 3 Degrees of Freedom (DoFs) is then estimated for a robot arm with a gripper for the segmented cloud of each instance of waste. Finally, two tactile-based algorithms are implemented and then employed in order to provide the gripper with a sense of touch. This work uses two low-cost visual-based tactile sensors at the fingertips. One of them addresses the detection of contact (which is obtained from tactile images) between the gripper and solid waste, while another has been designed to detect slippage in order to prevent the objects grasped from falling. Our proposal was successfully tested by carrying out extensive experimentation with different objects varying in size, texture, geometry and materials in different outdoor environments (a tiled pavement, a surface of stone/soil, and grass). Our system achieved an average score of 94% for the detection and Collection Success Rate (CSR) as regards its overall performance, and of 80% for the collection of items of litter at the first attempt.

Proceedings of the 5th International Symposium on Sensor Science (I3S 2017), 2017

Education in the Knowledge Society, 2012

This paper discusses different teaching experiences which aims are the learning robotics in the u... more This paper discusses different teaching experiences which aims are the learning robotics in the university. These experiences are reflected in the development of several robotics courses and subjects at the University of Alicante. The authors have created various educational platforms or they have used tools of free distribution and open source for the implementation of these courses. The main objetive of these courses is to teach the design and implementation of robotic solutions to solve various problems not only such as the control, programming and handling of robot but also the assembly, building and programming of educational mini-robots. On the one hand, new teaching tools are used such as simulators and virtual labs which make flexible the learning of robot arms. On the other hand, competitions are used to motivate students because this way, the students put into action the skills learned through building and programming low-cost mini-robots.

Latest trends in robotic grasping combine vision and touch for improving the performance of syste... more Latest trends in robotic grasping combine vision and touch for improving the performance of systems at tasks like stability prediction. However, tactile data are only available during the grasp, limiting the set of scenarios in which multimodal solutions can be applied. Could we obtain it prior to grasping? We explore the use of visual perception as a stimulus for generating tactile data so the robotic system can ”feel” the response of the tactile perception just by looking at the object.

Sensors (Basel, Switzerland), Jan 20, 2018

This paper presents a system that combines computer vision and surface electromyography technique... more This paper presents a system that combines computer vision and surface electromyography techniques to perform grasping tasks with a robotic hand. In order to achieve a reliable grasping action, the vision-driven system is used to compute pre-grasping poses of the robotic system based on the analysis of tridimensional object features. Then, the human operator can correct the pre-grasping pose of the robot using surface electromyographic signals from the forearm during wrist flexion and extension. Weak wrist flexions and extensions allow a fine adjustment of the robotic system to grasp the object and finally, when the operator considers that the grasping position is optimal, a strong flexion is performed to initiate the grasping of the object. The system has been tested with several subjects to check its performance showing a grasping accuracy of around 95% of the attempted grasps which increases in more than a 13% the grasping accuracy of previous experiments in which electromyograph...

IEEE Transactions on Geoscience and Remote Sensing, 2018

This paper presents a system for the detection of ships and oil spills using side-looking airborn... more This paper presents a system for the detection of ships and oil spills using side-looking airborne radar (SLAR) images. The proposed method employs a two-stage architecture composed of three pairs of convolutional neural networks (CNNs). Each pair of networks is trained to recognize a single class (ship, oil spill, and coast) by following two steps: a first network performs a coarse detection, and then, a second specialized CNN obtains the precise localization of the pixels belonging to each class. After classification, a postprocessing stage is performed by applying a morphological opening filter in order to eliminate small look-alikes, and removing those oil spills and ships that are surrounded by a minimum amount of coast. Data augmentation is performed to increase the number of samples, owing to the difficulty involved in obtaining a sufficient number of correctly labeled SLAR images. The proposed method is evaluated and compared to a single multiclass CNN architecture and to previous state-of-the-art methods using accuracy, precision, recall, F-measure, and intersection over union. The results show that the proposed method is efficient and competitive, and outperforms the approaches previously used for this task.

Learning and teaching processes are continually changing. Therefore, design of learning technolog... more Learning and teaching processes are continually changing. Therefore, design of learning technologies has gained interest in educators and educational institutions from secondary school to higher education. This paper describes the successfully use in education of social learning technologies and virtual laboratories designed by the authors, as well as videos developed by the students. These tools, combined with other open educational resources based on a blended-learning methodology, have been employed to teach the subject of Computer Networks. We have verified not only that the application of OERs into the learning process leads to a significantly improvement of the assessments, but also that the combination of several OERs enhances their effectiveness. These results are supported by, firstly, a study of both students ’ opinion and students ’ behaviour over five academic years, and, secondly, a correlation analysis between the use of OERs and the grades obtained by students.

Event-based visual servoing is a recently presented approach that performs the positioning of a r... more Event-based visual servoing is a recently presented approach that performs the positioning of a robot using visual information only when it is required. From the basis of the classical image-based visual servoing control law, the scheme proposed in this paper can reduce the processing time at each loop iteration in some specific conditions. The proposed control method enters in action when an event deactivates the classical image-based controller (i.e. when there is no image available to perform the tracking of the visual features). A virtual camera is then moved through a straight line path towards the desired position. The virtual path used to guide the robot improves the behavior of the previous event-based visual servoing proposal.

Both tactile and visual sensing are important to solve problems of uncertainties inherent to the ... more Both tactile and visual sensing are important to solve problems of uncertainties inherent to the grasping tasks. In this paper, a hybrid approach based on combining tactile and RGBD data is proposed for control the robotic manipulation of elastic objects. The proposed method detects both interaction and lack of contact among fingertips and object. Also, this approach measures deformations to determine appropriated finger movements when the object's geometry changes in real time as a result of the deformation caused by the pressure. This approach was evaluated on a multi-fingered robot hand with tactile sensors in real experiments.