A Deep Learning-Based Dirt Detection Computer Vision System for Floor-Cleaning Robots with Improved Data Collection (original) (raw)
Related papers
Cascaded Machine-Learning Technique for Debris Classification in Floor-Cleaning Robot Application
Applied Sciences, 2018
Debris detection and classification is an essential function for autonomous floor-cleaning robots. It enables floor-cleaning robots to identify and avoid hard-to-clean debris, specifically large liquid spillage debris. This paper proposes a debris-detection and classification scheme for an autonomous floor-cleaning robot using a deep Convolutional Neural Network (CNN) and Support Vector Machine (SVM) cascaded technique. The SSD (Single-Shot MultiBox Detector) MobileNet CNN architecture is used for classifying the solid and liquid spill debris on the floor through the captured image. Then, the SVM model is employed for binary classification of liquid spillage regions based on size, which helps floor-cleaning devices to identify the larger liquid spillage debris regions, considered as hard-to-clean debris in this work. The experimental results prove that the proposed technique can efficiently detect and classify the debris on the floor and achieves 95.5% percent classification accurac...
Sensors, 2020
Conveyor belts are the most widespread means of transportation for large quantities of materials in the mining sector. Therefore, autonomous methods that can help human beings to perform the inspection of the belt conveyor system is a major concern for companies. In this context, we present in this work a novel and automatic visual detector that recognizes dirt buildup on the structures of conveyor belts, which is one of the tasks of the maintenance inspectors. This visual detector can be embedded as sensors in autonomous robots for the inspection activity. The proposed system involves training a convolutional neural network from RGB images. The use of the transfer learning technique, i.e., retraining consolidated networks for image classification with our collected images has shown very effective. Two different approaches for transfer learning have been analyzed. The best one presented an average accuracy of 0.8975 with an F-1 Score of 0.8773 for the dirt recognition. A field valid...
A Computer Vision System for Street Sweeper Robot
International Journal of Advanced Computer Science and Applications
With the spread of Covid-19, more people wear personal protective equipment such as gloves and masks. However, they are littering them all over streets, parking lots and parks. This impacts the environment and damages especially the marine ecosystem. Thus, this waste should not be discarded in the environment. Moreover, it should not be recycled with other plastic materials. Actually, they have to be separated from regular trash collection. Furthermore, littering gloves and masks yields more workload for street cleaners and presents potential harm for them. In this paper, we design a computer vision system for a street sweeper robot that picks up the masks and gloves and disposes them safely in garbage containers. This system relies on Deep Learning techniques for object recognition. In particular, three Deep Learning models will be investigated. They are: You Only Look Once (YOLO) model, Faster Region based Convolutional Neural Network (Faster R-CNN) and DeepLab v3+. The experiment results showed that YOLO is the most suitable approach to design the proposed system. Thus, the performance of the proposed system is 0.94 as F1 measure, 0.79 as IoU, 0.94 as mAP, and 0.41 s as Time to process one image.
IRJET- A Deep Learning Approach for Real-Time Garbage's Detection and Cleanliness Assessment
IRJET, 2020
The ability of science and technology have improved in many fields for human life. But still, many real-time problems are a challenge in address about street cleanliness to city administrators. The city manager spends a great deal of effort and cash on street cleanliness. Consequently, street cleanliness is important for a healthy life. Therefore, existing methods have some of the cons, such as data gathering of street garbage's wasn't self-active, and no real-time cleanliness assessment. To overcome these cons, this paper proposes a deep learning approach for real-time garbage's detection and cleanliness assessment. Firstly, a camera installed on the street which collects the street images. secondly, processed information is transmitted to the data server to analyse in the network. during at the same time, Faster RCNN utilized to detect the street garbage's and classifying these categories and calculate the quantity of garbage's detected on frames. Finally, the result has visualized the garbage detected on the street, which convenience to city administrators to allot a clean-up process to the street effectively. The implementation of this approach will contribute to create the community a better place.
sTetro-Deep Learning Powered Staircase Cleaning and Maintenance Reconfigurable Robot
Sensors
Staircase cleaning is a crucial and time-consuming task for maintenance of multistory apartments and commercial buildings. There are many commercially available autonomous cleaning robots in the market for building maintenance, but few of them are designed for staircase cleaning. A key challenge for automating staircase cleaning robots involves the design of Environmental Perception Systems (EPS), which assist the robot in determining and navigating staircases. This system also recognizes obstacles and debris for safe navigation and efficient cleaning while climbing the staircase. This work proposes an operational framework leveraging the vision based EPS for the modular re-configurable maintenance robot, called sTetro. The proposed system uses an SSD MobileNet real-time object detection model to recognize staircases, obstacles and debris. Furthermore, the model filters out false detection of staircases by fusion of depth information through the use of a MobileNet and SVM. The syste...
Deep Learning Based Pavement Inspection Using Self-Reconfigurable Robot
Sensors
The pavement inspection task, which mainly includes crack and garbage detection, is essential and carried out frequently. The human-based or dedicated system approach for inspection can be easily carried out by integrating with the pavement sweeping machines. This work proposes a deep learning-based pavement inspection framework for self-reconfigurable robot named Panthera. Semantic segmentation framework SegNet was adopted to segment the pavement region from other objects. Deep Convolutional Neural Network (DCNN) based object detection is used to detect and localize pavement defects and garbage. Furthermore, Mobile Mapping System (MMS) was adopted for the geotagging of the defects. The proposed system was implemented and tested with the Panthera robot having NVIDIA GPU cards. The experimental results showed that the proposed technique identifies the pavement defects and litters or garbage detection with high accuracy. The experimental results on the crack and garbage detection are ...
YOLO-GD: A Deep Learning-Based Object Detection Algorithm for Empty-Dish Recycling Robots
Machines
Due to the workforce shortage caused by the declining birth rate and aging population, robotics is one of the solutions to replace humans and overcome this urgent problem. This paper introduces a deep learning-based object detection algorithm for empty-dish recycling robots to automatically recycle dishes in restaurants and canteens, etc. In detail, a lightweight object detection model YOLO-GD (Ghost Net and Depthwise convolution) is proposed for detecting dishes in images such as cups, chopsticks, bowls, towels, etc., and an image processing-based catch point calculation is designed for extracting the catch point coordinates of the different-type dishes. The coordinates are used to recycle the target dishes by controlling the robot arm. Jetson Nano is equipped on the robot as a computer module, and the YOLO-GD model is also quantized by TensorRT for improving the performance. The experimental results demonstrate that the YOLO-GD model is only 1/5 size of the state-of-the-art model ...
Skip-YOLO: Domestic Garbage Detection Using Deep Learning Method in Complex Multi-scenes
2021
In In the field of garbage intelligent identification, similar garbage are difficult to be effectively detected due to different kinds of characteristics. This paper proposes a Skip-YOLO model for garbage detection in real life through the visual analysis of feature mapping in different neural networks. First of all, the receptive field of the model is enlarged through the large-size convolution kernel, which enhanced the shallow information of images. Secondly, the high-dimensional feature mappings of garbage is extracted by dense convolutional blocks. The sensitivity of similar features in the same type of garbage is enhanced by strengthening the sharing of shallow low semantics and deep high semantics information. Finally, the multi-scale high-dimensional feature mappings is integrated and sent to the YOLO layer to predict the type and location of garbage. Experimental results show that compared with the YOLOv3, the overall detection precision is increased by 22.5%, and the avera...
2005
Cleaning of livestock buildings is the single most health-threatening task in the agricultural industry and a transition to robot-based cleaning would be instrumental to improving working conditions for employees. Present cleaning robots fall short on cleanness quality, as they cannot perform condition based cleaning. This paper describes how a novel sensor, developed for the purpose, and algorithms for classification and learning are combined with a commercial robot to obtain an autonomous system which meets the necessary quality attributes. These include features to make selective cleaning where dirty areas are detected, that operator assistance is called only when cleanness hypothesis cannot be made with confidence. The paper describes the design of the system where learning from experience maps and operator instructions are combined to obtain a smart and autonomous cleaning robot.
A Generic Deep-Learning-Based Approach for Automated Surface Inspection
IEEE Transactions on Cybernetics, 2017
Automated surface inspection (ASI) is a challenging task in industry, as collecting training dataset is usually costly and related methods are highly dataset-dependent. In this paper, a generic approach that requires small training data for ASI is proposed. First, this approach builds classifier on the features of image patches, where the features are transferred from a pretrained deep learning network. Next, pixel-wise prediction is obtained by convolving the trained classifier over input image. An experiment on three public and one industrial data set is carried out. The experiment involves two tasks: 1) image classification and 2) defect segmentation. The results of proposed algorithm are compared against several best benchmarks in literature. In the classification tasks, the proposed method improves accuracy by 0.66%-25.50%. In the segmentation tasks, the proposed method reduces error escape rates by 6.00%-19.00% in three defect types and improves accuracies by 2.29%-9.86% in all seven defect types. In addition, the proposed method achieves 0.0% error escape rate in the segmentation task of industrial data.