Reducing environment exposure to COVID-19 by IoT sensing and computing with deep learning (original) (raw)

Abstract

The COVID-19 pandemic has caused significant harm globally, prompting us to prioritize prevention measures. Effective hand-washing is one of the most critical and straightforward measures that can help prevent the spread of this virus. Medical staff’s hands are considered a major source of hospital infection. Effective hand-washing can prevent up to 30% of diarrhea-related illnesses, which is crucial in preventing nosocomial infections (Tartari et al. in Clin Microbiol Infect 23(9):596–598, 2017). This paper proposes an electronic-based real-time hand-washing identification framework called Alpha Hand Washing (ALPHA HW). The system uses camera-based identification, edge computing, and deep learning to automatically identify correct hand-washing behaviors, thereby facilitating effective hand-washing (Bertasius et al. in: Computer vision and pattern recognition, 2015). We achieved an accuracy of 78.0% mAP and a speed of 52 FPS in detecting scenes using specific monitoring datasets in hospitals by constructing the complex recognition system into a grid computing problem. Leveraging edge computing, our system achieves real-time identification with low memory consumption and high-efficiency computation. Alpha HW presents scientific and financial values in epidemic prevention and control that can facilitate popularization to reduce virus spread (Bewley et al. in 2016 IEEE international conference on image processing, 2016).

Access this article

Log in via an institution

Subscribe and save

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Data availability

The experiment data is provided by the third party for academic use under specific permission.

References

  1. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916
    Article Google Scholar
  2. Bertasius G, Shi J, Torresani L (2015) Deepedge: a multi-scale bifurcated deep network for top–down contour detection. In Computer vision and pattern recognition
  3. Bewley A, Ge Z, Ott L, Ramos F, Upcroft B (2016) Simple online and realtime tracking. In 2016 IEEE international conference on image processing (ICIP), pp 3464–3468. IEEE
  4. Bochkovskiy A, Wang CY, Mark LHY (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934
  5. Canny JF (1987) A computational approach to edge detection. Readings in computer vision. Morgan Kaufmann 1987:184–203
    Google Scholar
  6. Chen Jiasi, Ran Xukan (2019) Deep learning with edge computing: a review. Proc IEEE 107(8):1655–1674
    Article Google Scholar
  7. Centers for Disease Control and Prevention(CDC). When and how to wash your hands, (2021)
  8. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)
  9. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: IEEE transactions on pattern analysis and machine intelligence
  10. He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–16
    Article Google Scholar
  11. Hwang JJ, Liu TL (2015) Pixel-wise deep learning for contour detection. Comput Sci
  12. Kittler J (1983) On the accuracy of the sobel edge detector. Image Vis Comput 1(1):37–42
    Article Google Scholar
  13. Konishi S, Yuille AL, Coughlan JM, Song CZ (2003) Statistical edge detection: learning and evaluating edge cues. IEEE Trans Pattern Anal Mach Intell 25(1):57–74
    Article Google Scholar
  14. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR)
  15. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. IEEE
  16. Marr D, Hildreth E (1980) Theory of edge detection. Proc R Soc London 207(1167):187–217
    Google Scholar
  17. Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intell 26(5):530–549
    Article Google Scholar
  18. Moore Lori D, Greg Robbins, Jeff Quinn, Arbogast James W (2021) The impact of covid-19 pandemic on hand hygiene performance in hospitals. Am J Infect Control 49(1):30–33
    Article Google Scholar
  19. Pustokhina IV, Pustokhin DA, Gupta D, Khanna A, Shankar K, Nguyen GN (2020) An effective training scheme for deep neural network in edge computing enabled internet of medical things (iomt) systems. IEEE Access 8:107112–107123
    Article Google Scholar
  20. Wu X, Qi Y, Song J, Yao J, Wang Y, Liu Y, Han Y, Qian Q (2022) CA-STD: Scene text detection in arbitrary shape based on conditional attention. Information 13(12):565
    Article Google Scholar
  21. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. arXiv preprint arXiv:1506.01497
  22. Rothan HA, Byrareddy SN (2020) The epidemiology and pathogenesis of coronavirus disease (Covid-19) outbreak. J Autoimmun 109:102433
    Article Google Scholar
  23. Sahin Ahmet-Riza, Erdogan Aysegul, Agaoglu Pelin-Mutlu, Dineri Yeliz, Cakirci Ahmet-Yusuf, Senel Mahmut-Egemen, Okyay Ramazan-Azim, Tasdogan Ali-Muhittin (2020) 2019 novel coronavirus (covid-19) outbreak: a review of the current literature. EJMO 4(1):1–7
    Google Scholar
  24. Ma CD, Song J, Xu YB, Fan HW, Wu X, Sun T (2023) Vehicle-Based Machine Vision Approaches in Intelligent Connected System. In: IEEE Transactions on Intelligent Transportation Systems
  25. Tartari E, Abbas M, Pires D, De Kraker MEA, Pittet D (2017) World health organization save lives: clean your hands global campaign-‘fight antibiotic resistance-it’s in your hands’. Clin Microbiol Infect 23(9):596–598
  26. Wei S, Wang X, Yan W, Xiang B, Zhang Z (2015) Deepcontour: A deep convolutional feature learned by positive-sharing loss for contour detection. In: Computer Vision & Pattern Recognition
  27. Wojke Nicolai, Bewley Alex, Paulus Dietrich (2017) Simple online and realtime tracking with a deep association metric. In: 2017 IEEE international conference on image processing (ICIP), pp 3645–3649 IEEE
  28. Xie S, Tu Z (2015) Holistically-nested edge detection. Int J Comput Vis 125(1–3):3–18
    MathSciNet Google Scholar
  29. Zaidi SSA, Ansari MS, Aslam A, Kanwal N, Asghar M, Lee B (2022) A survey of modern deep learning based object detection models. Digit Signal Process 126:103514
    Article Google Scholar

Download references

Funding

The work is supported by the Smart Society Lab at Hong Kong Baptist University.

Author information

Authors and Affiliations

  1. Department of Geography, Hong Kong Baptist University, Kowloon Tong, China
    Chendong Ma & Jun Song
  2. Smart Society Lab, Hong Kong Baptist University, Kowloon Tong, China
    Jun Song
  3. Zhilun Technology Research Institute, Nanjing, China
    Yibo Xu
  4. Department of Natural Sciences, Imperial College London, London, UK
    Hongwei Fan
  5. Department of Electrical and Electronic Engineering, Imperial College London, London, UK
    Xiaoran Liu
  6. School of Computer Engineering and Science, Shanghai University, Shanghai, China
    Xing Wu
  7. Department of Computer Science, University of Electronic and Technology, Chengdu, China
    Yang Luo
  8. School of Traffic, Tongji University, Shanghai, China
    Tuo Sun
  9. Department of Civil Engineering, The University of Hong Kong, Hong Kong, China
    Jiemin Xie

Authors

  1. Chendong Ma
  2. Jun Song
  3. Yibo Xu
  4. Hongwei Fan
  5. Xiaoran Liu
  6. Xing Wu
  7. Yang Luo
  8. Tuo Sun
  9. Jiemin Xie

Corresponding author

Correspondence toJun Song.

Ethics declarations

Conflicts of interest

There are no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article

Ma, C., Song, J., Xu, Y. et al. Reducing environment exposure to COVID-19 by IoT sensing and computing with deep learning.Neural Comput & Applic 35, 25097–25106 (2023). https://doi.org/10.1007/s00521-023-08712-9

Download citation

Keywords