Traffic congestion-aware graph-based vehicle rerouting framework from aerial imagery (original) (raw)

Abstract

In digital era, being stranded from very basic telecommunication protocols and internet makes vehicle rerouting-like crucial tools more difficult or even impossible, especially in times of disaster and emergency. In this study, we propose a modular rerouting framework for only one single vehicle composed of visual perception, property estimation and trajectory optimization, which enables to generate optimum paths exploiting aerial imagery. Once the deep network, which is fine-tuned on newly introduced dataset herein named MaVefAI, processes the input, the following module estimates pose, motion direction and speed of detected vehicles. Afterwards, we link the appropriate vehicles via graphs to obtain group properties that pave the way for estimating the traffic congestion level. In the end, we get the output as the optimum path from the independent trajectory optimization module to which required inputs are already sent by preceding modules. We solve the multi-objective cost function subject to velocity and congestion intervals, which comprises distance, traffic congestion level, and angle inconsistency. We employ Dijkstra, A*, RRT, and RRT* to optimize the cost while vast majority of existing works focus to optimize via single method. The fine-tuned segmentation model accuracy becomes more than 98% for vehicle groups thanks to MaVefAI. The extensive experiments reveal that all algorithms follow the same path. However, RRT* achieves the fastest result by examining most of the possible options in less time, which also appears to be the most robust method comparing with the alternatives for route optimization. Our dataset MaVefAI is publicly available here: https://precisioncomputing.sakarya.edu.tr.

Loading...

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

References (97)

  1. Al-Smadi, M., Abdulrahim, K., Seman, K., Salam, R., 2020. A new spatio-temporal background-foreground bimodal for motion segmentation and detection in Urban traffic scenes. Neural Comput. Appl. 32, 9453-9469.
  2. AlKheder, S., Alkhamees, W., Almutairi, R., Alkhedher, M., 2021. Bayesian combined neural network for traffic volume short-term forecasting at adjacent intersections. Neural Comput. Appl. 33, 1785-1836.
  3. Amato, G., Carrara, F., Falchi, F., Gennaro, C., Meghini, C., Vairo, C., 2017. Deep learning for decentralized parking lot occupancy detection. Expert Syst. Appl. 72, 327-334.
  4. Arishi, A., Krishnan, K., Arishi, M., 2022. Machine learning approach for truck-drones based last-mile delivery in the era of industry 4.0. Eng. Appl. Artif. Intell. 116, 105439.
  5. Audebert, N., Le Saux, B., Lefèvre, S., 2017. Segment-before-detect: Vehicle detection and classification through semantic segmentation of aerial images. Remote Sens. 9, 368. Bando, M., Hasebe, K., Nakayama, A., Shibata, A., Sugiyama, Y., 1995. Dynamical model of traffic congestion and numerical simulation. Phys. Rev. E. 51, 1035.
  6. Bayraktar, E., Basarkan, M., Celebi, N., 2020. A low-cost UAV framework towards ornamental plant detection and counting in the wild. ISPRS J. Photogramm. Remote Sens. 167, 1-11.
  7. Bayraktar, E., Boyraz, P., 2017. Analysis of feature detector and descriptor combinations with a localization experiment for various performance metrics. Turk. J. Electr. Eng. Comput. Sci. 25.
  8. Bayraktar, E., Yigit, C., Boyraz, P., 2019. A hybrid image dataset toward bridging the gap between real and simulation environments for robotics. Mach. Vis. Appl. 30, 23-40.
  9. Benjdira, B., Koubaa, A., Azar, A., Khan, Z., Ammar, A., Boulila, W., 2022. TAU: A framework for video-based traffic analytics leveraging artificial intelligence and unmanned aerial systems. Eng. Appl. Artif. Intell. 114, 105095.
  10. Bertsimas, D., Patterson, S., 2000. The traffic flow management rerouting problem in air traffic control: A dynamic network flow approach. Transp. Sci. 34, 239-255.
  11. Betts, J., 1998. Survey of numerical methods for trajectory optimization. J. Guid. Control Dyn. 21, 193-207.
  12. Boarnet, M., Kim, E., Parkany, E., 1998. Measuring traffic congestion. Transp. Res. Rec. 1634, 93-99.
  13. Buric, M., Pobar, M., Ivasic-Kos, M., 2018. Ball detection using YOLO and mask R-CNN. In: 2018 International Conference on Computational Science and Computational Intelligence. CSCI, pp. 319-323.
  14. Cannarsa, P., Frankowska, H., Marchini, E.M., 2009. On Bolza optimal control problems with constraints.
  15. Cao, Z., Jiang, S., Zhang, J., Guo, H., 2016a. A unified framework for vehicle rerouting and traffic light control to reduce traffic congestion. IEEE Trans. Intell. Transp. Syst. 18, 1958-1973.
  16. Cao, L., Wang, C., Li, J., 2016b. Vehicle detection from highway satellite images via transfer learning. Inform. Sci. 366, 177-187.
  17. Chen, Y., Lv, Y., Li, Z., Wang, F., 2016. Long short-term memory model for traffic congestion prediction with online open data. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems. ITSC, pp. 132-137.
  18. Chen, Q., Song, Y., Zhao, J., 2021. Short-term traffic flow prediction based on improved wavelet neural network. Neural Comput. Appl. 33, 8181-8190.
  19. Chen, C., Wang, H., Yuan, F., Jia, H., Yao, B., 2020. Bus travel time prediction based on deep belief network with back-propagation. Neural Comput. Appl. 32, 10435-10449.
  20. Chen, M., Yu, X., Liu, Y., 2018. PCNN: Deep convolutional networks for short-term traffic congestion prediction. IEEE Trans. Intell. Transp. Syst. 19, 3550-3559.
  21. Corman, F., D'Ariano, A., Pacciarelli, D., Pranzo, M., 2010. A tabu search algorithm for rerouting trains during rail operations. Transp. Res. B 44, 175-192.
  22. Cui, H., Yuan, G., Liu, N., Xu, M., Song, H., 2020. Convolutional neural network for recognizing highway traffic congestion. J. Intell. Transp. Syst. 24, 279-289.
  23. D'Andrea, E., Marcelloni, F., 2017. Detection of traffic congestion and incidents from GPS trace analysis. Expert Syst. Appl. 73, 43-56.
  24. Deng, Z., Sun, H., Zhou, S., Zhao, J., Zou, H., 2017. Toward fast and accurate vehicle detection in aerial images using coupled region-based convolutional neural networks. IEEE J. Sel. Top. Appl. Earth Observ. Remote Sens. 10, 3652-3664.
  25. Dijkstra, E., et al., 1959. A note on two problems in connexion with graphs. Numer. Math. 1, 269-271.
  26. Duhamel, C., Lacomme, P., Prodhon, C., 2012. A hybrid evolutionary local search with depth first search split procedure for the heterogeneous vehicle routing problems. Eng. Appl. Artif. Intell. 25, 345-358.
  27. Dutta, A., Zisserman, A., 2019. The VIA annotation software for images, audio and video. In: Proceedings of The 27th ACM International Conference On Multimedia. http://dx.doi.org/10.1145/3343031.3350535.
  28. Elloumi, M., Dhaou, R., Escrig, B., Idoudi, H., Saidane, L., 2018. Monitoring road traffic with a UAV-based system. In: 2018 IEEE Wireless Communications And Networking Conference. WCNC, pp. 1-6.
  29. Fan, Q., Brown, L., Smith, J., 2016. A closer look at Faster R-CNN for vehicle detection. In: 2016 IEEE Intelligent Vehicles Symposium. IV, pp. 124-129.
  30. Florian, Ehemalige Schmidt, Data set for tracking vehicles in aerial image sequences. KIT AIS Vehicles Data Set. Available online: https://bit.ly/3lPy53Q, (Accessed 25 May 2021).
  31. Garcia-Aunon, P., Roldán, J., Barrientos, A., 2019. Monitoring traffic in future cities with aerial swarms: Developing and optimizing a behavior-based surveillance algorithm. Cogn. Syst. Res. 54, 273-286.
  32. Geiger, A., Lenz, P., Urtasun, R., 2012. Are we ready for autonomous driving? In: The KITTI Vision Benchmark Suite. Conference on Computer Vision and Pattern Recognition. CVPR.
  33. Hansen, I., 2018. Determination and evaluation of traffic congestion costs. Eur. J. Trans. Infrastructure Res. 1, 61-72.
  34. Hart, P., Nilsson, N., Raphael, B., 1968. A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cybern. 4, 100-107.
  35. He, K., Gkioxari, G., Dollár, P., Girshick, R., 2017. Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2961-2969.
  36. He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recogni- tion. In: Proceedings of The IEEE Conference On Computer Vision And Pattern Recognition. pp. 770-778.
  37. Hsieh, M., Lin, Y., Hsu, W., 2017. Drone-based object counting by spatially regularized regional proposal network. In: Proceedings of The IEEE International Conference On Computer Vision. pp. 4145-4153.
  38. Hu, X., Zhang, C., Zhang, G., Zhang, M., Li, H., Leeson, M., Liao, J., 2020. Finding the k shortest paths by ripple-spreading algorithms. Eng. Appl. Artif. Intell. 87, 103229.
  39. Huang, Z., Xia, J., Li, F., Li, Z., Li, Q., 2019. A peak traffic congestion prediction method based on bus driving time. Entropy 21, 709.
  40. Jie, K., Liu, S., Sun, X., 2022. A hybrid algorithm for time-dependent vehicle routing problem with soft time windows and Stochastic factors. Eng. Appl. Artif. Intell. 109, 104606.
  41. Juba, B., Le, H., 2019. Precision-recall versus accuracy and the role of large data sets. In: Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. pp. 4039-4048.
  42. Juliani, A., Berges, V., Vckay, E., Gao, Y., Henry, H., Mattar, M., Lange, D., 2018. Unity: A general platform for intelligent agents. arXiv Preprint arXiv:1809.02627.
  43. Ke, R., Li, Z., Tang, J., Pan, Z., Wang, Y., 2018. Real-time traffic flow parameter estimation from UAV video based on ensemble classifier and optical flow. IEEE Trans. Intell. Transp. Syst. 20, 54-64.
  44. Kim, N., Bodunkov, N., 2018. Automated decision making in road traffic monitoring by on-board unmanned aerial vehicle system. In: Computer Vision in Control Systems. Vol. 3. pp. 149-175.
  45. Kim, G., Ong, Y., Cheong, T., Tan, P., 2016. Solving the dynamic vehicle routing problem under traffic congestion. IEEE Trans. Intell. Transp. Syst. 17, 2367-2380.
  46. Koga, Y., Miyazaki, H., Shibasaki, R., 2018. A CNN-based method of vehicle detection from aerial images using hard example mining. Remote Sens. 10, 124.
  47. LaValle, S., 1998. Rapidly-exploring random trees: A new tool for path planning. (Citeseer).
  48. LaValle, S., Kuffner, Jr., J., 2001. Randomized kinodynamic planning. Int. J. Robot. Res. 20, 378-400.
  49. Lei, D., Cui, Z., Li, M., 2022. A dynamical artificial bee colony for vehicle routing problem with drones. Eng. Appl. Artif. Intell. 107, 104510.
  50. Li, C., Chen, M., 2013. Identifying important variables for predicting travel time of freeway with non-recurrent congestion with neural networks. Neural Comput. Appl. 23, 1611-1629.
  51. Li, M., Zhen, L., Wang, S., Lv, W., Qu, X., 2018. Unmanned aerial vehicle scheduling problem for traffic monitoring. Comput. Ind. Eng. 122, 15-23.
  52. Lin, T., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C., 2014. Microsoft coco: common objects in context. In: European Conference On Computer Vision. pp. 740-755.
  53. Liu, X., Peng, Z., Zhang, L., 2019. Real-time UAV rerouting for traffic monitoring with decomposition based multi-objective optimization. J. Intell. Robot. Syst. 94, 491-501.
  54. Manibardo, E., Laña, I., Lobo, J., Ser, J.Del., 2020. New perspectives on the use of online learning for congestion level prediction over traffic data. In: 2020 International Joint Conference On Neural Networks. IJCNN, pp. 1-8.
  55. Marinakis, Y., Marinaki, M., Dounias, G., 2010. A hybrid particle swarm optimization algorithm for the vehicle routing problem. Eng. Appl. Artif. Intell. 23, 463-472.
  56. Mou, L., Zhu, X., 2018. Vehicle instance segmentation from aerial image and video using a multitask learning residual fully convolutional network. IEEE Trans. Geosci. Remote Sens. 56, 6699-6711.
  57. Moyano, A., Stepniak, M., Moya-Gomez, B., Garcia-Palomares, J., 2021. Traffic con- gestion and economic context: changes of spatiotemporal patterns of traffic travel times during crisis and post-crisis periods. Transportation 1-24.
  58. Mundhenk, T., Konjevod, G., Sakla, W., Boakye, K., 2016. A large contextual dataset for classification, detection and counting of cars with deep learning. In: European Conference On Computer Vision. pp. 785-800.
  59. Nguyen, H., Lee, E., Bae, C., Lee, S., 2020. Multiple object detection based on clustering and deep learning methods. Sensors 20, 4424.
  60. Niemirepo, T., Toivonen, J., Viitanen, M., Vanne, J., 2019. Open-source CiThruS simulation environment for real-time 360-degree traffic imaging. In: 2019 IEEE International Conference On Connected Vehicles And Expo. ICCVE, pp. 1-5.
  61. Niu, H., Gonzalez-Prelcic, N., Heath, R., 2018. A UAV-based traffic monitoring system- invited paper. In: 2018 IEEE 87th Vehicular Technology Conference. VTC Spring, pp. 1-5.
  62. Officials, T., 2011. A policy on geometric design of highways and streets. (AASHTO, 2011).
  63. Oliveira, D., Rampinelli, M., Tozatto, G., Andreão, R., Müller, S., 2021. Forecasting vehicular traffic flow using MLP and LSTM. Neural Comput. Appl. 1-12.
  64. Pan, J., Popa, I., Zeitouni, K., Borcea, C., 2013. Proactive vehicular traffic rerouting for lower travel time. IEEE Trans. Veh. Technol. 62, 3551-3568.
  65. Pinto, L., Almeida, L., Alizadeh, H., Rowe, A., 2017. Aerial video stream over multi- hop using adaptive tdma slots. In: 2017 IEEE Real-Time Systems Symposium. RTSS pp. 157-166.
  66. Razakarivony, S., Jurie, F., 2016. Vehicle detection in aerial imagery: A small target detection benchmark. J. Vis. Commun. Image Represent. 34, 187-203.
  67. Ren, S., He, K., Girshick, R., Sun, J., 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems. pp. 91-99.
  68. Rocha, D., Aloise, D., Aloise, D., Contardo, C., 2022. Visual attractiveness in vehicle routing via bi-objective optimization. Comput. Oper. Res. 137, 105507.
  69. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al., 2015. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211-252.
  70. Sakla, W., Konjevod, G., Mundhenk, T., 2017. Deep multi-modal vehicle detection in aerial ISR imagery. In: 2017 IEEE Winter Conference on Applications of Computer Vision. WACV, pp. 916-923.
  71. Schaeffer, S., 2007. Graph clustering. Comput. Sci. Rev. 1, 27-64.
  72. Shah, S., Dey, D., Lovett, C., Kapoor, A., 2018. Airsim: High-fidelity visual and physical simulation for autonomous vehicles. Field Serv. Robot. 621-635.
  73. Shahinfar, S., Meek, P., Falzon, G., 2020. How many images do I need? Understanding how sample size per class affects deep learning model performance metrics for balanced designs in autonomous wildlife monitoring. Ecol. Inform. 57, 101085.
  74. Sommer, L., Schuchert, T., Beyerer, J., 2017. Fast deep vehicle detection in aerial images. In: 2017 IEEE Winter Conference on Applications of Computer Vision. WACV, pp. 311-319.
  75. Soon, K., Lim, J., Parthiban, R., 2019. Extended pheromone-based short-term traffic forecasting models for vehicular systems. Eng. Appl. Artif. Intell. 82, 60-75.
  76. Su, H., Wei, S., Liu, S., Liang, J., Wang, C., Shi, J., Zhang, X., 2020. HQ-ISNet: High-quality instance segmentation for remote sensing imagery. Remote Sens. 12, 989. Tajbakhsh, N., Shin, J., Gurudu, S., Hurst, R., Kendall, C., Gotway, M., Liang, J., 2016. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE Trans. Med. Imag. 35, 1299-1312.
  77. Tang, Z., Naphade, M., Liu, M., Yang, X., Birchfield, S., Wang, S., Kumar, R., Anastasiu, D., Hwang, J., 2019. Cityflow: A city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification. In: Proceedings of The IEEE Conference On Computer Vision And Pattern Recognition. pp. 8797-8806.
  78. Toda, Y., Okura, F., Ito, J., Okada, S., Kinoshita, T., Tsuji, H., Saisho, D., 2020. Training instance segmentation neural network with synthetic datasets for crop seed phenotyping. Commun. Biol. 3, 1-12.
  79. Tsitsulin, A., Palowitch, J., Perozzi, B., Müller, E., 2020. Graph clustering with graph neural networks. arXiv Preprint arXiv:2006.16904.
  80. Vahdani, B., Veysmoradi, D., Shekari, N., Mousavi, S., 2018. Multi-objective, multi- period location-routing model to distribute relief after earthquake by considering emergency roadway repair. Neural Comput. Appl. 30, 835-854.
  81. Wang, S., Djahel, S., Zhang, Z., McManis, J., 2016. Next road rerouting: A multiagent system for mitigating unexpected urban traffic congestion. IEEE Trans. Intell. Transp. Syst. 17, 2888-2899.
  82. Wang, W., Guo, R., Yu, J., 2018. Research on road traffic congestion index based on comprehensive parameters: Taking dalian city as an example. Adv. Mech. Eng. 10, 1687814018781482.
  83. Wu, X., Li, W., Hong, D., Tian, J., Tao, R., Du, Q., 2020. Vehicle detection of multi- source remote sensing data using active fine-tuning network. ISPRS J. Photogramm. Remote Sens. 167, 39-53.
  84. Wu, G., Mao, N., Luo, Q., Xu, B., Shi, J., Suganthan, P., 2022. Collaborative truck- drone routing for contactless parcel delivery during the epidemic. IEEE Trans. Intell. Transp. Syst..
  85. Wu, M., Zhang, G., Bi, N., Xie, L., Hu, Y., Shi, Z., Intelligence, Y., 2019. Multiview vehicle tracking by graph matching model. In: CVPR Workshops. pp. 29-36.
  86. Xia, G., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., Zhang, L., 2018. DOTA: A large-scale dataset for object detection in aerial images. In: Proceedings of The IEEE Conference On Computer Vision And Pattern Recognition. pp. 3974-3983.
  87. Xu, H., Jiang, C., Liang, X., Li, Z., 2019. Spatial-aware graph relation network for large-scale object detection. In: Proceedings of The IEEE Conference On Computer Vision And Pattern Recognition. pp. 9298-9307.
  88. Yang, Y., Liu, F., Wang, P., Luo, P., Liu, X., 2012. Vehicle detection methods from an unmanned aerial vehicle platform. In: 2012 IEEE International Conference on Vehicular Electronics and Safety. ICVES 2012, pp. 411-415.
  89. Yang, J., Xie, X., Yang, W., 2019. Effective contexts for UAV vehicle detection. IEEE Access 7, 85042-85054.
  90. Yanmaz, E., Quaritsch, M., Yahyanejad, S., Rinner, B., Hellwagner, H., Bettstetter, C., 2017. Communication and coordination for drone networks. Ad Hoc Netw. 79-91.
  91. Yasdi, R., 1999. Prediction of road traffic using a neural network approach. Neural Comput. Appl. 8, 135-142.
  92. Yu, D., Deng, L., Dahl, G., 2010. Roles of pre-training and fine-tuning in context- dependent DBN-HMMs for real-world speech recognition. In: Proc. NIPS Workshop On Deep Learning And Unsupervised Feature Learning.
  93. Zang, X., Jiang, L., Liang, C., Dong, J., Lu, W., Mladenovic, N., 2022. Optimization approaches for the urban delivery problem with trucks and drones. Swarm Evol. Comput. 75, 101147.
  94. Zhang, Q., Yin, C., Chen, Y., Su, F., 2022. IGCRRN: Improved graph convolution res-recurrent network for spatio-temporal dependence capturing and traffic flow prediction. Eng. Appl. Artif. Intell. 114, 105179.
  95. Zhao, P., Hu, H., 2019. Geographical patterns of traffic congestion in growing megacities: Big data analytics from Beijing. Cities 92, 164-174.
  96. Zhao, Z., Zheng, P., Xu, S., Wu, X., 2019. Object detection with deep learning: A review. IEEE Trans. Neural Netw. Learn. Syst. 30, 3212-3232.
  97. Zhu, X., Vondrick, C., Fowlkes, C., Ramanan, D., 2016. Do we need more training data? Int. J. Comput. Vis. 119, 76-92.