Distributed Q-learning for energy harvesting Heterogeneous Networks

We consider a two-tier urban Heterogeneous Network where small cells powered with renewable energy are deployed in order to provide capacity extension and to offload macro base stations. We use reinforcement learning techniques to concoct an algorithm that autonomously learns energy inflow and traffic demand patterns. This algorithm is based on a decentralized multi-agent Q-learning technique that, by interacting with the environment, obtains optimal policies aimed at improving the system performance in terms of drop rate, throughput and energy efficiency. Simulation results show that our solution effectively adapts to changing environmental conditions and meets most of our performance objectives. At the end of the paper we identify areas for improvement.