Reinforcement Learning-based User Association in Sustainable Terrestrial Non-Terrestrial 6G Networks

In sixth generation (6G) networks, efficient resource allocation will be pivotal to the viability of three-dimensional (3D) networks. In this paper, we study the online user association problem in an integrated Terrestrial and Non-Terrestrial (TN-NTN) 6G network, with the objective of maximizing energy efficiency subject to capacity and Quality of Service (QoS) constraints. First, we formulate the problem as a Mixed Integer Linear Program (MILP) that minimizes the total power consumption. We then formulate it as a Markov Decision Process (MDP) and propose a low complexity reinforcement-learning solution based on Proximal Policy Optimization (PPO) to make user association decisions under the same constraints. Extensive multiscenario simulations show that the proposed DNN-based policy achieves almost 10 times lower power consumption compared to the State-of-the-Art (SoA), while incurring significantly lower computational complexity than the optimal MILP, in a multinode network with more than 140 base stations.