Optimizing of Q-Learning Day/Night Energy Strategy for Solar Harvesting Environmental Wireless Sensor Networks Nodes

This research article presents the application of
the Q-learning algorithm in the operational duty cycle control
of solar-powered environmental wireless sensor network
(EWSN) nodes. Those nodes are commonly implemented as
embedded devices using low-power and low-cost
microcontrollers. Therefore, there is a significant need for an
effective and easy way to implement a machine learning (ML)
algorithm in terms of computer performance. This approach
uses a Q-learning-based policy implementing a sleep/run
switching algorithm driven by the state of charge. The
presented algorithm is based on two modes: daylight and
nighttime, which is a suitable solution for solar-powered
systems. The study includes the complete process of design
EWSN node strategy with an optimal reward policy. The
presented algorithm was tested and verified on an EWSN node
model and a 5-year data set of solar irradiance values was used
for the learning process and its validation. As part of the study,
we are also presenting the validation in terms of Q-learning
parameters, which include the learning rate and discount
factor. The result section shows that the overall performance of
the presented solution is more suitable for solar-powered
EWSN then state-of-the-art studies. Both day/night
experiments reached 828 203 measurement/transmission
cycles, which is 12.7 % more than in the previous studies using
the strategy defined by the state of energy storage.