Service Migration Optimization for System Overhead Minimization in VECNs via Deep Reinforcement Learning

In vehicular edge computing networks (VECNs), service migration among edge servers is critical to addressing the challenge of service interruption caused by high mobility of vehicles and limited coverage of each edge server. In this article, we tackle this challenge by optimizing service migration among edge servers through a joint management of resource scheduling and dynamic server selection. Specifically, we aim to minimize system overhead consisting of system time and energy consumption taking account for resource scheduling and dynamic server selection, which is formulated as a constrained optimization problem. To solve this optimization problem, we propose a learning-driven joint resource scheduling and dynamic server selection strategy (LD-JRS3) based on deep reinforcement learning. Under the LD-JRS3 strategy, we first model joint resource scheduling and dynamic server selection as a Markov decision process (MDP). Then, we adopt a recurrent neural network (RNN)-empowered feedback mechanism based on historical information to achieve the optimal system performance. We fully consider the advantages of the soft actor-critic (SAC) algorithm to obtain the optimal decision (i.e., computational resources allocation and servers selection). Notably, we employ an improved SAC algorithm, which takes into account prioritized experience replay and automatic tuning of temperature parameters. Extensive simulation results are presented to verify the effectiveness of our proposed LD-JRS3 algorithm, and also to illustrate the advantage of our algorithm on improving the time consumption and energy consumption compared with the baseline schemes. LD-JRS3 has 19%, 24%, and 11% higher utility values than DDRN, DQN-based, and multiarmed bandit-based systems, respectively.