Skip to content
2000
  1. Home
  2. A-Z Publications
  3. Recent Patents on Engineering
  4. Volume 19, Issue 5
  5. Article
Volume 19, Issue 5
  • ISSN: 1872-2121
  • E-ISSN: 2212-4047

Abstract

Background

In recent years, with the development of the Internet of Vehicles, a variety of novel in-vehicle application devices have surfaced, exhibiting increasingly stringent requirements for time delay. Vehicular edge networks (VEN) can fully use network edge devices, such as roadside units (RSUs), for collaborative processing, which can effectively reduce latency.

Objective

Most extant studies, including patents, assume that RSU has sufficient computing resources to provide unlimited services. But in fact, its computing resources will be limited with the increase in processing tasks, which will restrict the delay-sensitive vehicular applications. To solve this problem, a vehicle-to-vehicle computing task offloading method based on deep reinforcement learning is proposed in this paper, which fully considers the remaining available computational resources of neighboring vehicles to minimize the total task processing latency and enhance the offloading success rate.

Methods

In the multi-service vehicle scenario, the analytic hierarchy process (AHP) was first used to prioritize the computing tasks of user vehicles. Next, an improved sequence-to-sequence (Seq2Seq) computing task scheduling model combined with an attention mechanism was designed, and the model was trained by an actor-critic (AC) reinforcement learning algorithm with the optimization goal of reducing the processing delay of computing tasks and improving the success rate of offloading. A task offloading strategy optimization model based on AHP-AC was obtained on this basis.

Results

The average latency and execution success rate are used as performance metrics to compare the proposed method with three other task offloading methods: Only-local processing, greedy strategy-based algorithm, and random algorithm. In addition, experimental validation in terms of CPU frequency and the number of SVs is carried out to demonstrate the excellent generalization ability of the proposed method.

Conclusion

The simulation results reveal that the proposed method outperforms other methods in reducing the processing delay of tasks and improving the success rate of task offloading, which solves the problem of limited execution of delay-sensitive tasks caused by insufficient computational resources.

© 2025 Bentham Science Publishers
Loading

Article metrics loading...

/content/journals/eng/10.2174/0118722121283037231231064521
2024-01-16
2025-04-06

  • From This Site

    /content/journals/eng/10.2174/0118722121283037231231064521
    dcterms_title,dcterms_subject,pub_keyword
    -contentType:Contributor -contentType:Concept -contentType:Institution
    10
    5
Loading full text...

Full text loading...

References

  1. AlamI. KumarS. KashyapP.K. Alam1I. KumarS. Kumar KashyapP. A seven-layered model architecture, network model, protocol stack, security, application, issues and challenges in internet of vehicleRecent Pat. Eng.2021154e21042118390410.2174/1872212114999200719144002
     [Google Scholar]
  2. WollschlaegerM. SauterT. JasperneiteJ. The future of industrial communication: Automation networks in the era of the internet of things and industry 4.0.IEEE Ind. Electron. Mag.2017111172710.1109/MIE.2017.2649104
     [Google Scholar]
  3. SharmaS. KaushikB. A survey on internet of vehicles: Applications, security issues & solutions.Vehicular Communications20192010018210.1016/j.vehcom.2019.100182
     [Google Scholar]
  4. KumarM. RajuK.S. KumarD. GoyalN. VermaS. SinghA. An efficient framework using visual recognition for IoT based smart city surveillance.Multimedia Tools Appl.20218020312773129510.1007/s11042‑020‑10471‑x33495686
     [Google Scholar]
  5. ZhouH. XuW. ChenJ. WangW. Evolutionary V2X technologies toward the internet of vehicles: challenges and opportunities.Proc. IEEE2020108230832310.1109/JPROC.2019.2961937
     [Google Scholar]
  6. JainR. DhingraS. JoshiK. RanaA.K. GoyalN. Enhance traffic flow prediction with Real-Time Vehicle Data Integration.Journal of Autonomous Intelligence202362574https://api.semanticscholar.org/CorpusID:26083502910.32629/jai.v6i2.574
     [Google Scholar]
  7. BellendorfJ. MannZ.Á. Classification of optimization problems in fog computing.Future Gener. Comput. Syst.202010715817610.1016/j.future.2020.01.036
     [Google Scholar]
  8. ScellatoS. FortunaL. FrascaM. Gómez-GardeñesJ. LatoraV. Traffic optimization in transport networks based on local routing.Eur. Phys. J. B201073230330810.1140/epjb/e2009‑00438‑2
     [Google Scholar]
  9. ZhangZ. WangN. WuH. TangC. LiR. MR-DRO: A fast and efficient task offloading algorithm in heterogeneous edge/cloud computing environments.IEEE Internet Things J.20231043165317810.1109/JIOT.2021.3126101
     [Google Scholar]
  10. SadatdiynovK. A review of optimization methods for computation offloading in edge computing networks.Digit. Commun. and Netw.20229245046110.1016/j.dcan.2022.03.003
     [Google Scholar]
  11. DaiY. XuD. MaharjanS. ZhangY. ZhangY. Joint load balancing and offloading in vehicular edge computing and networks.IEEE Internet Things J.2019634377438710.1109/JIOT.2018.2876298
     [Google Scholar]
  12. MaoY. YouC. ZhangJ. HuangK. LetaiefK.B. A survey on mobile edge computing: the communication perspective.IEEE Commun. Surv. Tutor.20171942322235810.1109/COMST.2017.2745201
     [Google Scholar]
  13. AbbasN. ZhangY. TaherkordiA. SkeieT. Mobile edge computing: a survey.IEEE Internet Things J.20185145046510.1109/JIOT.2017.2750180
     [Google Scholar]
  14. LiuJ. LinF. LiuK. ZhaoY. LiJ. Research on multi-terminal’s AC offloading scheme and multi-server’s AC selection scheme in IoT.Entropy (Basel)202224101357135710.3390/e2410135737420377
     [Google Scholar]
  15. SaccoA. EspositoF. MarchettoG. MontuschiP. Sustainable task offloading in UAV networks via multi-agent reinforcement learning.IEEE Trans. Vehicular Technol.20217055003501510.1109/TVT.2021.3074304
     [Google Scholar]
  16. WangJ. FengD. ZhangS. LiuA. XiaX-G. Joint computation offloading and resource allocation for MEC-enabled IoT systems with imperfect CSI.IEEE Internet Things J.2021853462347510.1109/JIOT.2020.3022802
     [Google Scholar]
  17. HuS. LiG. Dynamic request scheduling optimization in mobile edge computing for IoT applications.IEEE Internet Things J.2020721426143710.1109/JIOT.2019.2955311
     [Google Scholar]
  18. HuangL. BiS. ZhangY-J.A. ZhangY.J. Deep reinforcement learning for online computation offloading in wireless powered mobile-edge computing networks.IEEE Trans. Mobile Comput.202019112581259310.1109/TMC.2019.2928811
     [Google Scholar]
  19. LinL. LiaoX. JinH. LiP. Computation offloading toward edge computing.Proc. IEEE201910781584160710.1109/JPROC.2019.2922285
     [Google Scholar]
  20. ElgendyI.A. YadavR. Survey on mobile edge-cloud computing: A taxonomy on computation offloading approaches.Studies in Big Data20229511715810.1007/978‑3‑030‑85428‑7_6
     [Google Scholar]
  21. YuZ. XuG. LiY. LiuP. LiL. Joint offloading and energy harvesting design in multiple time blocks for FDMA based wireless powered MEC.Future Internet20211337010.3390/fi13030070
     [Google Scholar]
  22. SaeikF. AvgerisM. SpatharakisD. SantiN. DechouniotisD. ViolosJ. LeivadeasA. AthanasopoulosN. MittonN. PapavassiliouS. Task offloading in Edge and Cloud Computing: A survey on mathematical, artificial intelligence and control theory solutions.Comput. Netw.202119510817710.1016/j.comnet.2021.108177
     [Google Scholar]
  23. XiaoS. LiuC. LiK. LiK. System delay optimization for Mobile Edge Computing.Future Gener. Comput. Syst.2020109172810.1016/j.future.2020.03.028
     [Google Scholar]
  24. SharmaA. GoyalN. GuleriaK. Performance optimization in delay tolerant networks using backtracking algorithm for fully credits distribution to contrast selfish nodes.J. Supercomput.20217766036605510.1007/s11227‑020‑03507‑4
     [Google Scholar]
  25. ChenX. CaiY. LiL. ZhaoM. ChampagneB. HanzoL. Energy-efficient resource allocation for latency-sensitive mobile edge computing.IEEE Trans. Vehicular Technol.20206922246226210.1109/TVT.2019.2962542
     [Google Scholar]
  26. ShiH. LuoR. GuiG. GuiG. GuiG. Joint offloading and energy optimization for wireless powered mobile edge computing under nonlinear EH Model.Peer-to-Peer Netw. Appl.20211442248226110.1007/s12083‑021‑01172‑9
     [Google Scholar]
  27. DziyauddinR.A. NiyatoD. LuongN.C. Mohd AtanA.A.A. Mohd IzharM.A. AzmiM.H. Mohd DaudS. Computation offloading and content caching and delivery in Vehicular Edge Network: A survey.Comput. Netw.2021197510822810.1016/j.comnet.2021.108228
     [Google Scholar]
  28. ZhangG. GuX. LiJ. GuJ. ChenJ. JiY. DuanW. Energy-efficient optimized computing offloading method for ve-hicular edge computing network and system thereof.U.S. Patent 202202106862022
     [Google Scholar]
  29. LuH. GuC. LuoF. DingW. ZhengS. ShenY. Optimization of task offloading strategy for mobile edge computing based on multi-agent deep reinforcement learning.IEEE Access2020820257320258410.1109/ACCESS.2020.3036416
     [Google Scholar]
  30. HouC. ZhaoQ. Optimal task-offloading control for edge computing system with tasks offloaded and computed in sequence.IEEE Trans. Autom. Sci. Eng.20232021378139210.1109/TASE.2022.3176745
     [Google Scholar]
  31. LiuL. ChangZ. GuoX. MaoS. RistaniemiT. Multiobjective optimization for computation offloading in fog computing.IEEE Internet Things J.20185128329410.1109/JIOT.2017.2780236
     [Google Scholar]
  32. CaiP. YangF. WangJ. WuX. YangY. LuoX. JOTE: Joint offloading of tasks and energy in fog-enabled IoT networks.IEEE Internet Things J.2020743067308210.1109/JIOT.2020.2964951
     [Google Scholar]
  33. AlfakihT. HassanM.M. GumaeiA. SavaglioC. FortinoG. Task offloading and resource allocation for mobile edge computing by deep reinforcement learning based on SARSA.IEEE Access20208540745408410.1109/ACCESS.2020.2981434
     [Google Scholar]
  34. ZhangH. YangY. HuangX. FangC. ZhangP. Ultra-low latency multi-task offloading in mobile edge computing.IEEE Access20219325693258110.1109/ACCESS.2021.3061105
     [Google Scholar]
  35. RenJ. MahfujulK.M. LyuF. YueS. ZhangY. Joint channel allocation and resource management for stochastic computation offloading in MEC.IEEE Trans. Vehicular Technol.20206988900891310.1109/TVT.2020.2997685
     [Google Scholar]
  36. WangP. YaoC. ZhengZ. SunG. SongL. SongL. Joint task assignment, transmission, and computing resource allocation in multilayer mobile edge computing systems.IEEE Internet Things J.2019622872288410.1109/JIOT.2018.2876198
     [Google Scholar]
  37. JiangC. ChengX. GaoH. ZhouX. WanJ. WanJ. Toward computation offloading in edge computing: A survey.IEEE Access2019713154313155810.1109/ACCESS.2019.2938660
     [Google Scholar]
  38. MaoS. HeS. WuJ. Joint UAV position optimization and resource scheduling in space-air-ground integrated networks with mixed cloud-edge computing.IEEE Syst. J.20211533992400210.1109/JSYST.2020.3041706
     [Google Scholar]
  39. CaoZ. ZhouP. LiR. HuangS. WuD. Multiagent deep reinforcement learning for joint multichannel access and task offloading of mobile-edge computing in industry 4.0.IEEE Internet Things J.2020776201621310.1109/JIOT.2020.2968951
     [Google Scholar]
  40. MengA. WeiG. ZhaoY. GaoX. YangZ. Green resource allocation for mobile edge computing.Digit. Commun. Netw.2022951190119910.1016/j.dcan.2022.03.001
     [Google Scholar]
  41. WangK. ZhouY. LiuZ. ShaoZ. LuoX. YangY. Online task scheduling and resource allocation for intelligent noma-based industrial internet of things.IEEE J. Sel. Areas Comm.202038580381510.1109/JSAC.2020.2980908
     [Google Scholar]
  42. MitsisG. ApostolopoulosP.A. TsiropoulouE.E. PapavassiliouS. Intelligent dynamic data offloading in a competitive mobile edge computing market.Future Internet201911511810.3390/fi11050118
     [Google Scholar]
  43. BiS. HuangL. WangH. ZhangY-J.A. Lyapunov-guided deep reinforcement learning for stable online computation offloading in mobile-edge computing networks.IEEE Trans. Wirel. Commun.202120117519753710.1109/TWC.2021.3085319
     [Google Scholar]
  44. MaoS. LengS. MaharjanS. ZhangY. ZhangY. Energy efficiency and delay tradeoff for wireless powered mobile-edge computing systems with multi-access schemes.IEEE Trans. Wirel. Commun.20201931855186710.1109/TWC.2019.2959300
     [Google Scholar]
  45. WangY. TaoX. HouY.T. ZhangP. HouY.T. ZhangP. Effective capacity-based resource allocation in mobile edge computing with two-stage tandem queues.IEEE Trans. Commun.20196796221623310.1109/TCOMM.2019.2920835
     [Google Scholar]
  46. LiuX. YuJ. WangJ. GaoY. GaoY. Resource allocation with edge computing in IoT networks via machine learning.IEEE Internet Things J.2020743415342610.1109/JIOT.2020.2970110
     [Google Scholar]
  47. ShiJ. DuJ. WangJ. WangJ. YuanJ. Distributed V2V computation offloading based on dynamic pricing using deep reinforcement learning.IEEE Trans. Vehicular Technol.20206912160671608110.1109/TVT.2020.3041929
     [Google Scholar]
  48. LiuY. YuH. XieS. ZhangY. Deep reinforcement learning for offloading and resource allocation in vehicle edge computing and networks.IEEE Trans. Vehicular Technol.20196811111581116810.1109/TVT.2019.2935450
     [Google Scholar]
  49. LangP. TianD. DuanX. ZhouJ. ShengZ. LeungV.C.M. Cooperative computation offloading in blockchain-based vehicular edge computing networks.IEEE Trans. Intell. Veh.20227378379810.1109/TIV.2022.3190308
     [Google Scholar]
  50. MaC. ZhuJ. LiuM. ZhaoH. LiuN. ZouX. Parking edge computing: Parked vehicle assisted task offloading for urban VANETs.IEEE Internet Things J.20218119344935810.1109/JIOT.2021.3056396
     [Google Scholar]
  51. DaiX. XiaoZ. JiangH. ChenH. MinG. DustdarS. CaoJ. A learning based approach for vehicle-to-vehicle computation offloading.IEEE Internet Things J.20231087244725810.1109/JIOT.2022.3228811
     [Google Scholar]
  52. ZhangJ. GuoH. LiuJ. ZhangY. Task offloading in vehicular edge computing networks: A load-balancing solution.IEEE Trans. Vehicular Technol.20206922092210410.1109/TVT.2019.2959410
     [Google Scholar]
  53. TranT.X. PompiliD. Joint task offloading and resource allocation for multi-server mobile-edge computing networks.IEEE Trans. Vehicular Technol.201968185686810.1109/TVT.2018.2881191
     [Google Scholar]
  54. WangJ. ZhuK. ChenB. HanZ. Distributed clustering based cooperative vehicular edge computing for real-time offloading requests.IEEE Trans. Vehicular Technol.202271165366910.1109/TVT.2021.3122001
     [Google Scholar]
  55. ChenW. YaguchiY. NaruseK. WatanobeY. NakamuraK. QoS-aware robotic streaming workflow allocation in cloud robotics systems.IEEE Trans. Serv. Comput.202114254455810.1109/TSC.2018.2803826
     [Google Scholar]
  56. GautamV. TrivediN.K. SinghA. MohamedH.G. NoyaI.D. KaurP. GoyalN. A transfer learning-based artificial intelligence model for leaf disease assessment.Sustainability20221420136101362910.3390/su142013610
     [Google Scholar]
  57. ChenX. ZhangH. WuC. MaoS. JiY. BennisM. Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning.IEEE Internet Things J.2019634005401810.1109/JIOT.2018.2876279
     [Google Scholar]
  58. KarimiE. ChenY. AkbariB. Task offloading in vehicular edge computing networks via deep reinforcement learning.Comput. Commun.2022189119320410.1016/j.comcom.2022.04.006
     [Google Scholar]
  59. TangH. WuH. QuG. LiR. Efficient task scheduling for servers with dynamic states in vehicular edge computing.IEEE Trans. Netw. Sci. Eng.20231031297131010.1109/TNSE.2022.3172794
     [Google Scholar]
  60. GaoJ. KuangZ. GaoJ. ZhaoL. Joint offloading scheduling and resource allocation in vehicular edge computing: A two layer solution.IEEE Trans. Vehicular Technol.20237233999400910.1109/TVT.2022.3220571
     [Google Scholar]
  61. WangX. XingH. SongF. LuoS. DaiP. ZhaoB. On jointly optimizing partial offloading and SFC mapping: A cooperative dual-agent deep reinforcement learning approach.IEEE Trans. Parallel Distrib. Syst.20233482479249710.1109/TPDS.2023.3287633
     [Google Scholar]
  62. HuangH. YeQ. ZhouY. Deadline-aware task offloading with partially-observable deep reinforcement learning for multi-access edge computing.IEEE Trans. Netw. Sci. Eng.2022963870388510.1109/TNSE.2021.3115054
     [Google Scholar]
  63. JuY. ChenY. CaoZ. LiuL. PeiQ. XiaoM. OtaK. DongM. LeungV.C.M. Joint secure offloading and resource allocation for vehicular edge computing network: A multi-agent deep reinforcement learning approach.IEEE Trans. Intell. Transp. Syst.20232455555556910.1109/TITS.2023.3242997
     [Google Scholar]
  64. PengX. HanZ. XieW. YuC. ZhuP. XiaoJ. YangJ. Deep reinforcement learning for shared offloading strategy in vehicle edge computing.IEEE Syst. J.20231722089210010.1109/JSYST.2022.3190926
     [Google Scholar]
  65. WangJ. HuJ. MinG. ZhanW. ZomayaA.Y. GeorgalasN. Dependent task offloading for edge computing based on deep reinforcement learning.IEEE Trans. Comput.202271102449246110.1109/TC.2021.3131040
     [Google Scholar]
  66. XingY. SunY. QiaoL. WangZ. SiP. ZhangY. Deep reinforcement learning for cooperative edge caching in vehicular networks2021 13th International Conference on Communication Software and Networks (ICCSN)144149202110.1109/ICCSN52437.2021.9463666
     [Google Scholar]
  67. ZhuF. XuB. QiuJ. Unmanned aerial vehicle edge computing network linear de-pendency task unloading method.CN Patent 114599102
     [Google Scholar]
  68. ZhaoH. ZhangT. ChenY. Task distribution offloading algorithm of vehicle edge network based on DQN.J. Commun.20204110172178
     [Google Scholar]
  69. HouX. RenZ. WangJ. ChengW. RenY. ChenK-C. ZhangH. Reliable computation offloading for edge computing enabled software defined IoV.IEEE Internet Things J.2020787097711110.1109/JIOT.2020.2982292
     [Google Scholar]
  70. WilliamsR. Simple statistical gradient-following algorithms for connection-ist reinforcement learning.Mach. Learn.19928322925610.1007/BF00992696
     [Google Scholar]
  71. LiuL. CheC. FengJ. PeiQ. HeC. DouZ. Joint intelligent optimization of task offloading and service caching for vehicular edge computing.J. Commun.20214211826
     [Google Scholar]
/content/journals/eng/10.2174/0118722121283037231231064521
Loading
Computing Task Offloading in Vehicular Edge Network via Deep Reinforcement Learning
Recent Pat Eng 19, E160124225717 (2025); https://doi.org/10.2174/0118722121283037231231064521
/content/journals/eng/10.2174/0118722121283037231231064521
/content/journals/eng/10.2174/0118722121283037231231064521
Loading

Data & Media loading...


  • Article Type:     Research Article
Keyword(s): deep reinforcement learning; edge computing; priority; task offloading; task scheduling; Vehicular edge network

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test