Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks (original) (raw)

Abstract

Lack of a fully vehicular topology view and restricted vehicles' movement to streets with the time-varying traffic light conditions have caused drastic gaps in the traditional vehicular routing protocols. Using software-defined networks (SDN), this paper proposes HIFS, a Hierarchical Intersection-based routing strategy that incorporates Fuzzy SARSA reinforcement learning to fill these gaps. At the first level of our HIFS scheme, a utility-based intersections selection policy is presented using fuzzy logic that jointly considers delay estimation, curve distance, and predicted of moving vehicles towards intersections. Then, a fuzzy logic-based path selection policy is proposed to choose the paths with highest flexibility against the intermittent connectivity and increased traffic loads. Residual bandwidth, Euclidean distance, angular orientation, and congestion are considered inputs of the fuzzy logic system. Meanwhile, traffic light states and nodes' information are used to tune the output fuzzy membership functions via reinforcement learning algorithm. The efficiency of our scheme in controlling ambiguity and uncertainty of the vehicular environment is confirmed through simulations in various vehicle densities and different traffic lights duration. Simulation results of average gains obtained for both scenarios show that our HIFS scheme increases the packet delivery ratio on average by 48.75%, 54.63%, and 8.78%, increases the throughput by 48.66%, 53.79%, and 8.61%, reduces end-to-end delay by 33.35%, 46.14%, and 15.38%, reduces the path length by 25.25%, 36.47%, and 15.32%, and reduces normalized routing overhead by 37.09%, 49.79%, and 20.17%, compared to MISR, ITAR-FQ, and GLS methods, respectively.

Access this article

Log in via an institution

Subscribe and save

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Data availability

Not applicable.

References

  1. Al-Qurabat AKM, Mohammed ZA, Hussein ZJ (2021) Data traffic management based on compression and MDL techniques for smart agriculture in IoT. Wireless Pers Commun 120(3):2227–2258
    Google Scholar
  2. Al-Qurabat AKM (2021) A lightweight huffman-based differential encoding lossless compression technique in IoT for smart agriculture. Int J Comput Digital Syst 11(1). https://doi.org/10.12785/ijcds/110109
  3. Saeedi IDI, Al-Qurabat AKM (2022) Perceptually important points-based data aggregation method for wireless sensor networks. Baghdad Sci J 19(4):0875–0875
    Google Scholar
  4. Al-Qurabat AKM, Abdulzahra SA (2020) An overview of periodic wireless sensor networks to the internet of things. IOP Conf Ser: Mater Sci Eng 928(3):32055. https://doi.org/10.1088/1757-899X/928/3/032055
  5. Al-Qurabat AKM, Abdulzahra SA, Idrees AK (2022) Two-level energy-efficient data reduction strategies based on SAX-LZW and hierarchical clustering for minimizing the huge data conveyed on the internet of things networks. J Supercomput. 78:17844–17890. https://doi.org/10.1007/s11227-022-04548-7
  6. Al-Qurabat AKM, Idress AK, Makhoul A, Jooude CA (2022) A bi-level data lowering method to minimize transferring big data in the sensors of IoT applications. Karbala Int J Modern Sci 8(2):123–138
    Google Scholar
  7. Jawad GAM, Al-Qurabat AKM, Idrees AK (2022) Maximizing the underwater wireless sensor networks’ lifespan using BTC and MNP5 compression techniques. Ann Telecommun 77:703–723. https://doi.org/10.1007/s12243-021-00903-6
  8. Al-Qurabat AKM, Salman HM, Finjan AAR (2022) Important extrema points extraction-based data aggregation approach for elongating the WSN lifetime. Int J Comput Appl Technol 68(4):357–368
    Google Scholar
  9. Naderi M, Ghanbari M (2023) Adaptively prioritizing candidate forwarding set in opportunistic routing in VANETs. Ad Hoc Netw 140:03048. https://doi.org/10.1016/j.adhoc.2022.103048
  10. Cunha F, Villas L, Boukerche A, Maia G, Viana A, Mini RAF, Loureiro AAF (2016) Data communication in VANETs: Survey, applications and challenges. Ad Hoc Netw 44:90–103
    Google Scholar
  11. Islam MMd, R-Khan MT, Saad MM, Kim D (2021) Software-defined vehicular network (SDVN): a survey on architecture and routing. J Syst Archit 114:101961
    Google Scholar
  12. Zemouri S, Djahel S, Murphy J (2015) A fast, reliable and lightweight distributed dissemination protocol for safety messages in urban vehicular networks. Ad Hoc Netw 27:26–43
    Google Scholar
  13. Qadri NN, Fleury M, Altaf M, Ghanbari M (2010) Multi-source video streaming in a wireless vehicular ad hoc network. IET Commun 4(11):1300–1311
    Google Scholar
  14. Srivastava A, Prakash A, Tripathi R (2020) Location based routing protocols in VANET: Issues and existing solutions. Veh Commun 23:100231
    Google Scholar
  15. Al-Heety OS, Zakaria O, Ismail M, Shakir MM, Alani S, Alsariera H (2020) A comprehensive survey: Benefits, services, recent works, challenges, security and use cases for SDN-VANET. IEEE Access 8:91028–91047
    Google Scholar
  16. Naderi M, Zargari F, Sadatpour V, Ghanbari M (2017) A 3-parameter routing cost function for improving opportunistic routing performance in VANETs. Wireless Pers Commun 97(1):1–15
    Google Scholar
  17. Darwish T, Abu Bakar K (2016) Traffic aware routing in vehicular ad hoc networks: Characteristics and challenges. Telecommun Syst 61(3):489–513
    Google Scholar
  18. Darwish T, Abu Bakar K (2016) Lightweight intersection-based traffic aware routing in urban vehicular networks. Comput Commun 87:60–75
    Google Scholar
  19. Abdel-Halim IT, Fahmy HMA (2018) Prediction-based protocols for vehicular ad hoc networks: Survey and taxonomy. Comput Netw 130:34–50
    Google Scholar
  20. Jaballah WB, Conti M, Lal C (2019) Software-defined VANETs: Benefits, challenges, and future directions. Preprint at http://arxiv.org/abs/1904.04577
  21. Kadhim AJ, H-Seno SA (2019) Energy-efficient multicast routing protocol based on SDN and fog computing for vehicular networks. Ad Hoc Netw 85:68–81
    Google Scholar
  22. Zhao L, Zhao W, Hawbani A, Al-Dubai AY, Min G, Zomaya AY, Gong C (2021) Novel online sequential learning-based adaptive routing for edge software-defined vehicular networks. IEEE Trans Wireless Commun 20(5):2991–3004
    Google Scholar
  23. Oubbati OS, Atiquzzaman M, Lorenz P, Baz A, Alhakami H (2020) SEARCH: an SDN-enabled approach for vehicle path-planning. IEEE Trans Veh Technol 69(12):14523–14536
    Google Scholar
  24. Noorani N, H-Seno SA (2020) SDN and fog computing-based switchable routing using path stability estimation for vehicular ad hoc networks. Peer Peer Netw Appl 13(3):948–964
    Google Scholar
  25. Zhao L, Bi Zh, Lin M, Hawbani A, Shi J, Guan Y (2021) An intelligent fuzzy-based routing scheme for software-defined vehicular networks. Comput Netw 187:107837
    Google Scholar
  26. Abbas MT, Muhammad A, Song WC (2020) SD-IoV: SDN enabled routing for internet of vehicles in road-aware approach. J Ambient Intell Humaniz Comput 11(3):1265–1280
    Google Scholar
  27. Venkatramana DKN, Srikantaiah SB, Moodabidri J (2017) SCGRP: SDN-enabled connectivity-aware geographical routing protocol of VANETs for urban environment. IET Netw 6(5):102–111
    Google Scholar
  28. Kong D, Zhang G (2020) Ant colony algorithm-based routing protocol in software defined vehicular networks. In: Proceedings of the 2020 the 4th International Conference on Innovation in Artificial Intelligence, ICIAI 2020, Association for Computing Machinery, New York, NY, USA, pp 200–204
  29. Gao Y, Zhang Z, Zhao D, Zhang Y, Luo T (2018) A hierarchical routing scheme with load balancing in software defined vehicular Ad hoc networks. IEEE Access 6:73774–73785
    Google Scholar
  30. Chahal M, Harit S (2019) Network selection and data dissemination in heterogeneous software-defined vehicular network. Comput Netw 161:32–44
    Google Scholar
  31. Balta M, Ozçelik İ (2020) A 3-stage fuzzy-decision tree model for traffic signal optimization in urban city via a SDN based VANET architecture. Futur Gener Comput Syst 104:142–158
    Google Scholar
  32. Hagras H (2007) Type-2 FLCs: a new generation of fuzzy controllers. IEEE Comput Intell Mag 2(1):30–43
    Google Scholar
  33. Mardani MR, Ghanbari M (2018) Robust resource allocation scheme under channel uncertainties for LTE-A systems. Wireless Netw 25(3):1313–1325
    Google Scholar
  34. Ding Q, Sun B, Zhang X (2016) A traffic-light-aware routing protocol based on street connectivity for urban vehicular Ad hoc networks. IEEE Commun Lett 20(8):1635–1638
    Google Scholar
  35. Chang J-J, Li Y-H, Liao W, Chang C (2012) Intersection-based routing for urban vehicular communications with traffic-light considerations. IEEE Wirel Commun 19(1):82–88
    Google Scholar
  36. Xia Y, Qin X, Liu B, Zhang P (2018) A greedy traffic light and queue aware routing protocol for urban VANETs. China Commun 15(7):77–87
    Google Scholar
  37. Zhou S, Li D, Tang Q, Fu Y, Guo C, Chen X (2021) Multiple intersection selection routing protocol based on road section connectivity probability for urban VANETs. Comput Commun 177:255–264
    Google Scholar
  38. Cao Z, Silva BN, Diyan M, Li J, Han K (2020) Intersection routing based on fuzzy multi-factor decision for VANETs. Appl Sci 10(18):6613
    Google Scholar
  39. Cao Z, Fan Z, Kim J (2021) Intersection-based routing with fuzzy multi-factor decision for VANETs. Appl Sci 11(16):7304
    Google Scholar
  40. He T, He R, Yu C (2021) Intersection-based traffic-aware routing with Fuzzy Q-learning for urban VANETs. In: 2021 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), pp 511–515
  41. Debnath A, Basumatary H, Dhar M, Debbarma MK, Bhattacharyya BK (2021) Fuzzy logic-based VANET routing method to increase the QoS by considering the dynamic nature of vehicles. Computing 103(7):1391–1415
    Google Scholar
  42. Naderi M, Zargari F, Ghanbari M (2019) Adaptive beacon broadcast in opportunistic routing for VANETs. Ad Hoc Netw 86:119–130
    Google Scholar
  43. Alzamzami O, Mahgoub I (2020) Link utility aware geographic routing for urban VANETs using two-hop neighbor information. Ad Hoc Netw 106:102213
    Google Scholar
  44. Luo L, Sheng L, Yu H, Sun G (2021) Intersection-based V2X routing via reinforcement learning in vehicular ad hoc networks. IEEE Trans Intell Transp Syst 23(6):5446–5459. https://doi.org/10.1109/TITS.2021.3053958
  45. Srivastava A, Prakash A, Tripathi R (2020) Fuzzy-based beaconless probabilistic broadcasting for information dissemination in urban VANET. Ad Hoc Netw 108:102285
    Google Scholar
  46. Huynh T-T, Dinh-Duc A-V, Tran C-H (2016) Delay-constrained energy-efficient cluster-based multi-hop routing in wireless sensor networks. J Commun Netw 18(4):580–588
    Google Scholar
  47. Tal I, Muntean GM (2017) Towards reasoning vehicles: a survey of fuzzy logic-based solutions in vehicular networks. ACM Comput Surv (CSUR) 50(6):1–37
    Google Scholar
  48. Kim B-S, Ullah S, Kim KH, Roh B-S, Ham J-H, Kim K-I (2020) an enhanced geographical routing protocol based on multi criteria decision making method in mobile ad-hoc networks. Ad Hoc Netw 103:102157
    Google Scholar
  49. Derhami V, J-Majd V, N-Ahmadabadi M (2008) Fuzzy Sarsa learning and the proof of existence of its stationary points. Asian J Control 10(5):535–549
    MathSciNet Google Scholar
  50. Wu T, Zhou P, Liu K, Yuan Y, Wang X, Huang H, Wu DO (2020) Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks. IEEE Trans Veh Technol 69(8):8243–8256
    Google Scholar
  51. Rak J (2013) LLA: a new anypath routing scheme providing long path lifetime in VANETs. IEEE Commun Lett 18(2):281–284
    Google Scholar
  52. Zemouri S, Djahel S, Murphy J (2018) An altruistic prediction-based congestion control for strict beaconing requirements in urban VANETs. IEEE Trans Syst Man Cybern Syst 49(12):2582–2597
    Google Scholar
  53. NS-3 Homepage. [Online]. Available at https://www.nsnam.org. Accessed 25 Jul 2019
  54. Krajzewicz D, Hertkorn G, Rossel C, Wagner P (2002) SUMO (Simulation of Urban MObility), an open-source traffic simulation. In: Proc 4th MESM, Sharjah, UAE, pp 183–187
  55. Saleet H, Basir O, Langar R, Boutaba R (2009) Region-based location-service-management protocol for VANETs. IEEE Trans Veh Technol 59(2):917–931
    Google Scholar

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
    Mohammad Naderi & Khorshid Mahdaee
  2. Department of Computer Engineering, Pardis Branch, Islamic Azad University, Pardis, Iran
    Parisa Rahmani

Authors

  1. Mohammad Naderi
  2. Khorshid Mahdaee
  3. Parisa Rahmani

Contributions

All authors contributed equally to this manuscript.

Corresponding author

Correspondence toMohammad Naderi.

Ethics declarations

Ethics approval

The paper is original, and any other publishing house is not considering it for publication. The paper reflects the author's own research and analysis in a truthful and complete manner. All sources used are properly disclosed (correct citation).

Not applicable.

Not applicable.

Conflicts of interest

There is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article

Naderi, M., Mahdaee, K. & Rahmani, P. Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks.Peer-to-Peer Netw. Appl. 16, 1174–1198 (2023). https://doi.org/10.1007/s12083-022-01424-2

Download citation

Keywords