A low-cost UAV framework towards ornamental plant detection and counting in the wild (original) (raw)

Abstract

Abstract Object detection still keeps its role as one of the fundamental challenges within the computer vision territory. In particular, achieving satisfying results concerning object detection from outdoor images occupies a considerable space. In this study, in addition to comparing handcrafted feature detector/descriptor performance with deep learning methods over ornamental plant images at the outdoor, we propose a framework to improve the detection of these plants. Firstly, we take query images in the RGB format from the onboard UAV camera. Secondly, our model classifies the scene as a planting or an urban area. Thirdly, if the images are from planting area, thirdly, we filter the field according to the color and acquire only the green parts. Lastly, we feed the object detector model with the filtered area and obtain the category and localization of the plants as a result. In parallel, we also estimate the number of interested plants using the geometrical relations and predefined average plant size, then we verify the outputs of the object detector with this results. The conducted experiments show that deep learning based object detection methods overtake conventional feature detector/descriptor techniques in terms of accuracy, recall, precision, and sensitivity rates. The field classifier model, VGGNet, achieves a 98.17 % accuracy for this task, whilst YoloV3 achieves 91.6 % accuracy with 0.12 IOU for object detection as the best method. The proposed framework also improves the overall performance of these algorithms by 1.27 % for accuracy and 0.023 for IOU. By specifying the limits thoroughly and developing task-dependent approaches, we reveal the great potential of our framework plant detection and counting in the wild consisting of basic image preprocessing techniques, geometrical operations, and deep neural network.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (56)

  1. Ammour, Nassim, Alhichri, Haikel, Bazi, Yakoub, Benjdira, Bilel, Alajlan, Naif, Zuair, Mansour, 2017. Deep learning approach for car detection in uav imagery. Remote Sens. 9 (4), 312.
  2. Bay, Herbert, Tuytelaars, Tinne, Van Gool, Luc, 2006. Surf: Speeded up robust features. In: European Conference on Computer Vision. Springer, pp. 404-417.
  3. Bayraktar, Ertugrul, Boyraz, Pinar, 2017. Analysis of feature detector and descriptor combinations with a localization experiment for various performance metrics. Turk. J. Electric. Eng. Comput. Sci. 25 (3), 2444-2454.
  4. Berni, Jose A.J., Zarco-Tejada, Pablo J., Suárez, Lola, Fereres, Elias, 2009. Thermal and narrowband multispectral remote sensing for vegetation monitoring from an un- manned aerial vehicle. IEEE Trans. Geosci. Remote Sens. 47 (3), 722-738.
  5. Blaschke, Thomas, 2010. Object based image analysis for remote sensing. ISPRS J. Photogramm. Remote Sens. 65 (1), 2-16.
  6. Brunelli, Roberto, Poggio, Tomaso, 1993. Face recognition: features versus templates. IEEE Trans. Pattern Anal. Mach. Intell. 15 (10), 1042-1052.
  7. Candiago, Sebastian, Remondino, Fabio, De Giglio, Michaela, Dubbini, Marco, Gattelli, Mario, 2015. Evaluating multispectral images and vegetation indices for precision farming applications from uav images. Remote Sens. 7 (4), 4026-4047.
  8. Chen, D, Stow, D.A., Gong, P., 2004. Examining the effect of spatial resolution and texture window size on classification accuracy: an urban environment case. Int. J. Remote Sens. 25(11), 2177-2192.
  9. Colomina, Ismael, Molina, Pere, 2014. Unmanned aerial systems for photogrammetry and remote sensing: a review. ISPRS J. Photogramm. Remote Sens. 92, 79-97.
  10. Cruz, Henry O., Eckert, Martina, Meneses, Juan M., Fernán Martínez, José, 2016. Precise real-time detection of nonforested areas with uavs. IEEE Trans. Geosci. Remote Sens. 55 (2), 632-644.
  11. Deng, Jia, Dong, Wei, Socher, Richard, Li, Li-Jia, Li, Kai, Fei-Fei, Li, 2009. Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. Ieee, pp. 248-255.
  12. Everingham, Mark, Van Gool, Luc, Williams, Christopher K.I., Winn, John, Zisserman, Andrew, 2010. The pascal visual object classes (voc) challenge. Int. J. Comput. Vision 88 (2), 303-338.
  13. Fan, Zhun, Jiewei, Lu., Gong, Maoguo, Xie, Honghui, Goodman, Erik D, 2018. Automatic tobacco plant detection in uav images via deep neural networks. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 11 (3), 876-887.
  14. Gnädinger, Friederike, Schmidhalter, Urs, 2017. Digital counts of maize plants by un- manned aerial vehicles (uavs). Remote Sens. 9 (6), 544.
  15. Gracia-Romero, Adrian, Vergara-Díaz, Omar, Thierfelder, Christian, Cairns, Jill E., Kefauver, Shawn C., Araus, José L., 2018. Phenotyping conservation agriculture management effects on ground and aerial remote sensing assessments of maize hy- brids performance in zimbabwe. Remote Sens. 10 (2), 349.
  16. Grohmann, Carlos H., 2015. Effects of spatial resolution on slope and aspect derivation for regional-scale analysis. Comput. Geosci. 77, 111-117.
  17. Gueguen, Lionel, Pesaresi, Martino, Gerhardinger, Andrea, Soille, Pierre, 2011. Characterizing and counting roofless buildings in very high resolution optical images. IEEE Geosci. Remote Sens. Lett. 9 (1), 114-118.
  18. He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, Sun, Jian, 2016. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778.
  19. Hiary, Hazem, Saadeh, Heba, Saadeh, Mah., Yaqub, Mohammad, 2018. Flower classifi- cation using deep convolutional neural networks. IET Comput. Vision 12 (6), 855-862.
  20. Huang, Yanbo, Zhong-xin Chen, Y.U., Xiang-zhi Huang, Tao, Gu, Xing-fa, 2018. Agricultural remote sensing big data: management and applications. J. Integr. Agric. 17 (9), 1915-1931.
  21. Kang, Jian, Körner, Marco, Wang, Yuanyuan, Taubenböck, Hannes, Zhu, Xiao Xiang, 2018. Building instance classification using street view images. ISPRS J. Photogramm. Remote Sens. 145, 44-59.
  22. Kheirkhah, Fateme Mostajer, Asghari, Habibollah, 2018. Plant leaf classification using gist texture features. IET Comput. Vision 13 (4), 369-375.
  23. Krizhevsky, Alex, Sutskever, Ilya, Hinton, Geoffrey E., 2012. Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097-1105.
  24. Lakhal, Mohamed Ilyes, Çevikalp, Hakan, Escalera, Sergio, Ofli, Ferda, 2018. Recurrent neural networks for remote sensing image classification. IET Comput. Vision 12 (7), 1040-1045.
  25. LeCun, Yann, Bengio, Yoshua, Hinton, Geoffrey, 2015. Deep learning. Nature 521 (7553), 436-444.
  26. Li, Bo, Xu, Xiangming, Han, Jiwan, Zhang, Li, Bian, Chunsong, Jin, Liping, Liu, Jiangang, 2019. The estimation of crop emergence in potatoes by uav rgb imagery. Plant Meth. 15 (1), 15.
  27. Lin, Tsung-Yi, Maire, Michael, Belongie, Serge, Hays, James, Perona, Pietro, Ramanan, Deva, Dollár, Piotr, Lawrence Zitnick, C., 2014. Microsoft coco: common objects in context. In: European Conference on Computer Vision. Springer, pp. 740-755.
  28. Lin, Tsung-Yi, Goyal, Priya, Girshick, Ross, He, Kaiming, Dollár, Piotr, 2017. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980-2988.
  29. Liu, Ting, Yang, Xiaojun, 2015. Monitoring land changes in an urban area using satellite imagery, gis and landscape metrics. Appl. Geogr. 56, 42-54.
  30. Liu, Miao, Yu, Tao, Gu, Xingfa, Sun, Zhensheng, Yang, Jian, Zhang, Zhouwei, Mi, Xiaofei, Cao, Weijia, Li, Juan, 2020. The impact of spatial resolution on the classification of vegetation types in highly fragmented planting areas based on unmanned aerial ve- hicle hyperspectral images. Remote Sens. 12 (1), 146.
  31. Lowe, David G., 1999. In: Object Recognition from Local Scale-invariant Features, vol. 2. Ieee, pp. 1150-1157.
  32. Ma, Lei, Liu, Yu, Zhang, Xueliang, Ye, Yuanxin, Yin, Gaofei, Johnson, Brian Alan, 2019. Deep learning in remote sensing applications: a meta-analysis and review. ISPRS J. Photogramm. Remote Sens. 152, 166-177.
  33. Mansouri, Sina Sharif, Kanellakis, Christoforos, Georgoulas, George, Kominiak, Dariusz, Gustafsson, Thomas, Nikolakopoulos, George, et al., 2018. 2D visual area coverage and path planning coupled with camera footprints. Control Eng. Pract. 75, 1-16. https://doi.org/10.1016/j.conengprac.2018.03.011.
  34. Marpu, P.R., Neubert, M., Herold, H., Niemeyer, I., 2010. Enhanced evaluation of image segmentation results. J. Spatial Sci. 55 (1), 55-68.
  35. Moranduzzo, Thomas, Melgani, Farid, 2013. Automatic car counting method for un- manned aerial vehicle images. IEEE Trans. Geosci. Remote Sens. 52 (3), 1635-1647.
  36. Moranduzzo, Thomas, Melgani, Farid, 2014. Detecting cars in uav images with a catalog- based approach. IEEE Trans. Geosci. Remote Sens. 52 (10), 6356-6367.
  37. Muhammad, Usman, Wang, Weiqiang, Hadid, Abdenour, Pervez, Shahbaz, 2019. Bag of words kaze (bowk) with two-step classification for high-resolution remote sensing images. IET Comput. Vision 13 (4), 395-403.
  38. Muja, Marius, Lowe, David G., 2009. Fast approximate nearest neighbors with automatic algorithm configuration. VISAPP (1), 2(331-340):2.
  39. National Research Council, 1997. Precision Agriculture in the 21st Century: Geospatial and Information Technologies in Crop Management. The National Academies Press, Washington, DC.
  40. Nebiker, S., Lack, N., Abächerli, M., Läderach, S., 2016. Light-weight multispectral uav sensors and their capabilities for predicting grain yield and detecting plant diseases. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 41.
  41. Neubeck, Alexander, Van Gool, Luc, 2006. In: Efficient Non-maximum Suppression, vol.
  42. IEEE, pp. 850-855.
  43. Nguyen, Khang, Huynh, Nhut T., Nguyen, Phat C., Nguyen, Khanh-Duy, Vo, Nguyen D., Nguyen, Tam V., 2020. Detecting objects from space: an evaluation of deep-learning modern approaches. Electronics 9 (4), 583.
  44. Ok, Ali Ozgun, Senaras, Caglar, Yuksel, Baris, 2012. Automated detection of arbitrarily shaped buildings in complex environments from monocular vhr optical satellite imagery. IEEE Trans. Geosci. Remote Sens. 51(3), 1701-1717.
  45. Redmon, Joseph, Farhadi, Ali, 2018. Yolov3: An incremental improvement. arXiv pre- print arXiv:1804.02767.
  46. Rominger, Kody, Meyer, Susan E, 2019. Application of uav-based methodology for census of an endangered plant species in a fragile habitat. Remote Sens. 11 (6), 719.
  47. Rublee, Ethan, Rabaud, Vincent, Konolige, Kurt, Bradski, Gary, 2011. Orb: An efficient alternative to sift or surf. In: 2011 International Conference on Computer Vision. Ieee, pp. 2564-2571.
  48. Russakovsky, Olga, Deng, Jia, Hao, Su., Krause, Jonathan, Satheesh, Sanjeev, Ma, Sean, Huang, Zhiheng, Karpathy, Andrej, Khosla, Aditya, Bernstein, Michael, et al., 2015. Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115 (3), 211-252.
  49. Simonyan, Karen, Zisserman, Andrew, 2014. Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1556.
  50. Tao, Chao, Qi, Ji, Li, Yansheng, Wang, Hao, Li, Haifeng, 2019. Spatial information in- ference net: Road extraction using road-specific contextual information. ISPRS J. Photogramm. Remote Sens. 158, 155-166.
  51. Vakalopoulou, Maria, Karantzalos, Konstantinos, Komodakis, Nikos, Paragios, Nikos, 2015. Building detection in very high resolution multispectral data with deep learning features. In: 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). IEEE, pp. 1873-1876.
  52. von Bueren, Stefanie K, Burkart, Andreas, Hueni, Andreas, Rascher, Uwe, Tuohy, Mike P, Yule, Ian, 2015. Deploying four optical uav-based sensors over grassland: challenges and limitations. Biogeosciences 12 (1), 163-175.
  53. Xiang, Haitao, Tian, Lei, 2011. Development of a low-cost agricultural remote sensing system based on an autonomous unmanned aerial vehicle (uav). Biosyst. Eng. 108 (2), 174-190.
  54. Yang, Jun, Jiang, Yu-Gang, Hauptmann, Alexander G., Ngo, Chong-Wah, 2007. Evaluating bag-of-visual-words representations in scene classification. In: Proceedings of the International Workshop on Multimedia Information Retrieval, pp. 197-206.
  55. Zhang, Dongyan, Zhou, Xingen, Zhang, Jian, Lan, Yubin, Xu, Chao, Liang, Dong, 2018. Detection of rice sheath blight using an unmanned aerial system with high-resolution color and multispectral imaging. PloS One 13 (5).
  56. Zhong, Yanfei, Wang, Xinyu, Xu, Yao, Wang, Shaoyu, Jia, Tianyi, Hu, Xin, Zhao, Ji, Wei, Lifei, Zhang, Liangpei, 2018. Mini-uav-borne hyperspectral remote sensing: from observation and processing to applications. IEEE Geosci. Remote Sens. Mag. 6 (4), 46-62.