A3C-based load-balancing solution for computation offloading in SDN-enabled vehicular edge computing networks (original) (raw)

Abstract

In the Internet of vehicles (IoV), various applications emerging to provide users with safe, reliable, and comfortable driving services have caused explosive demand for computing capability. It is very challenging for vehicles with limited resources to meet the real-time demand. Mobile Edge Computing (MEC) is a convenient solution that utilizes computing resources at the network edge to enhance the capability of vehicles. With the help of MEC, vehicles can offload parts of tasks to nearby MEC servers to reduce the response cost. However, with the increase of vehicles, the unbalanced load among servers may lead to the unbalanced resource allocation and the degradation of task completion ratio. To address this problem, we first propose an SDN-enabled vehicular edge computing network architecture to facilitate the management of IoV with a global view. Then, the joint optimization problem of computation offloading and load balancing is formulated as a sequential decision problem to minimize the system cost in terms of time and energy. An A3C-based solution is proposed to solve the offloading decisions. Further, we develop the simple computation resource allocation scheme and calculation method of offloading ratio, respectively. Finally, the simulation results demonstrate the superior performance of our proposed algorithm compared to the benchmark solutions.

Access this article

Log in via an institution

Subscribe and save

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Zhou H, Xu W, Chen J, Wang W (2020) Evolutionary v2x technologies toward the internet of vehicles: Challenges and opportunities. Proc IEEE 108(2):308–323
    Article Google Scholar
  2. Sun Y, Guo X, Sheng Z, Jiang Z, Niu Z (2018) Learning-based task offloading for vehicular cloud computing systems. In: 2018 IEEE International Conference on Communications (ICC 2018), pp 1–7
  3. Shi J, Du J, Wang J, Wang J, Yuan J (2020) Priority-aware task offloading in vehicular fog computing based on deep reinforcement learning. IEEE Trans Veh Technol 69(12):16067–16081
    Article Google Scholar
  4. Zhang K, Mao Y, Leng S, Vinel A, Zhang Y (2016) Delay constrained offloading for mobile edge computing in cloud-enabled vehicular networks. In: 2016 8th International Workshop on Resilient Networks Design and Modeling (RNDM), pp 288–294
  5. Guo H, Liu J, Ren J, Zhang Y (2020) Intelligent task offloading in vehicular edge computing networks. IEEE Wirel Commun 27(4):126–132
    Article Google Scholar
  6. Khayyat M, Elgendy IA, Muthanna A, Alshahrani AS, Alharbi S, Koucheryavy A (2020) Advanced deep learning-based computational offloading for multilevel vehicular edge-cloud computing networks. IEEE Access 8:137052–137062
    Article Google Scholar
  7. Tang D, Zhang X, Li M, Tao X (2020) Adaptive inference reinforcement learning for task offloading in vehicular edge computing systems. In: 2020 IEEE International Conference on Communications Workshops (ICC Workshops), pp 1–6
  8. Wang J, Feng D, Zhang S, Tang J, Quek TQS (2019) Computation offloading for mobile edge computing enabled vehicular networks. IEEE Access 7:62624–62632
    Article Google Scholar
  9. Sun J, Gu Q, Zheng T, Dong P, Valera A, Qin Y (2020) Joint optimization of computation offloading and task scheduling in vehicular edge computing networks. IEEE Access 8:10466–10477
    Article Google Scholar
  10. Wang R, Zeng F, Deng X, Wu J (2021) Joint computation offloading and resource allocation in vehicular edge computing based on an economic theory: walrasian equilibrium. Peer-to-Peer Netw Appl 14:3971–3983
  11. Zhao J, Li Q, Gong Y, Zhang K (2019) Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks. IEEE Trans Veh Technol 68(8):7944–7956
    Article Google Scholar
  12. Ke H, Wang J, Deng L, Ge Y, Wang H (2020) Deep reinforcement learning-based adaptive computation offloading for mec in heterogeneous vehicular networks. IEEE Trans Veh Technol 69(7):7916–7929
    Article Google Scholar
  13. Peng H, Shen X (2020) Deep reinforcement learning based resource management for multi-access edge computing in vehicular networks. IEEE Trans Netw Sci Eng 7(4):2416–2428
    Article MathSciNet Google Scholar
  14. Zhan W, Luo C, Wang J, Wang C, Min G, Duan H, Zhu Q (2020) Deep-reinforcement-learning-based offloading scheduling for vehicular edge computing. IEEE Internet Things J 7(6):5449–5465
    Article Google Scholar
  15. Beraldi R, Mtibaa A, Alnuweiri H (2017) Cooperative load balancing scheme for edge computing resources. In: 2017 Second International Conference on Fog and Mobile Edge Computing (FMEC), pp 94–100
  16. Song N, Gong C, An X, Zhan Q (2016) Fog computing dynamic load balancing mechanism based on graph repartitioning. China Communications 13(3):156–164
    Article Google Scholar
  17. Wu H, Chen L, Shen C, Wen W, Xu J (2018) Online geographical load balancing for energy-harvesting mobile edge computing. In: 2018 IEEE International Conference on Communications (ICC), pp 1–6
  18. Truong NB, Lee GM, Ghamri-Doudane Y (2015) Software defined networking-based vehicular adhoc network with fog computing. In: 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM), pp 1202–1207
  19. He Y, Liang C, Zhang Z, Yu FR, Zhao N, Yin H, Zhang Y (2017) Resource allocation in software-defined and information-centric vehicular networks with mobile edge computing. In: 2017 IEEE 86th Vehicular Technology Conference (VTC-Fall), pp 1–5
  20. Xia W, Zhang J, Quek TQS, Jin S, Zhu H (2020) Mobile edge cloud-based industrial internet of things: Improving edge intelligence with hierarchical sdn controllers. IEEE Veh Technol Mag 15(1):36–45
    Article Google Scholar
  21. Zhao L, Zhao W, Hawbani A, Al-Dubai AY, Min G, Zomaya AY, Gong C (2021) Novel online sequential learning-based adaptive routing for edge software-defined vehicular networks. IEEE Trans Wirel Commun 20(5):2991–3004
    Article Google Scholar
  22. Hou X, Ren Z, Wang J, Cheng W, Ren Y, Chen KC, Zhang H (2020) Reliable computation offloading for edge-computing-enabled software-defined iov. IEEE Internet Things J 7(8):7097–7111
    Article Google Scholar
  23. Khadir A, Seno S (2021) Sdn-based offloading policy to reduce the delay in fog-vehicular networks. Peer-to-Peer Netw Appl 14:1261–1275
  24. Misra S, Bera S (2020) Soft-van: Mobility-aware task offloading in software-defined vehicular network. IEEE Trans Veh Technol 69(2):2071–2078
    Article Google Scholar
  25. Zhang H, Wang Z, Liu K (2020) V2x offloading and resource allocation in sdn-assisted mec-based vehicular networks. China Commun 17(5):266–283
    Article Google Scholar
  26. Volodymyr M, Adriá PB, Mehdi M, Alex G, Koray K (2016) Asynchronous methods for deep reinforcement learning. In: 2016 Proceedings of the 33rd International Conference on International Conference on Machine Learning, pp 1928–1937
  27. Ye Q, Zhuang W, Zhang S, Jin AL, Shen X, Li X (2018) Dynamic radio resource slicing for a two-tier heterogeneous wireless network. IEEE Trans Veh Technol 67(10):9896–9910
    Article Google Scholar
  28. IEEE (2011) IEEE draft standard for information technology - telecommunications and information exchange between systems - local and metropolitan area networks - specific requirements - part 11: wireless lan medium access control (mac) and physical layer (phy) specifications. IEEE P80211-REVmb/D12, November 2011, pp 1–2910

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China (Grant No.61771002); the National Education Ministry of China (No.NGII20170636); and the Fundamental Research Funds for the Central Universities (No.2021CZ102).

Author information

Authors and Affiliations

  1. School of Software Engineering, Beijing Jiaotong University, Beijing, 100044, China
    Lingyu Lu
  2. School of Computer and Information Technology, Beijing Jiaotong University, Beijing, 100044, China
    Jing Yu & Xiang Li
  3. Beijing Sankuai Online Technology Co., Ltd, 100102, Beijing, China
    Haifeng Du

Authors

  1. Lingyu Lu
  2. Jing Yu
  3. Haifeng Du
  4. Xiang Li

Corresponding author

Correspondence toJing Yu.

Ethics declarations

Conflicts of interest

The authors declare no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article

Lu, L., Yu, J., Du, H. et al. A3C-based load-balancing solution for computation offloading in SDN-enabled vehicular edge computing networks.Peer-to-Peer Netw. Appl. 16, 1242–1256 (2023). https://doi.org/10.1007/s12083-022-01412-6

Download citation

Keywords