Optimizing Information Freshness in Constrained IoT Systems: A Token-Based Approach

In Internet of Things (IoT) status update systems, where information is sampled and subsequently transmitted from a source to a destination node, the imperative necessity lies in maintaining the timeliness of information and updating the system with optimal frequency. Optimizing information freshness in resource-limited status update systems often involves Constrained Markov Decision Process (CMDP) problems with update rate constraints. Solving CMDP problems, especially with multiple constraints, is a challenging task. To address this, we present a token-based approach that transforms CMDP into an unconstrained MDP, simplifying the solution process. To demonstrate the comprehensiveness and effectiveness of the token-based
approach, we apply this method to systems with one and two update rate constraints to optimize two distinct metrics: Age of Incorrect Information (AoII) and Age of Information (AoI), respectively, and explore the analytical and numerical aspects. Additionally, we introduce an iterative triangle bisection method
for solving the CMDP problems with two constraints, comparing its results with the token-based MDP approach. The structure of the optimal token-based solution is studied analytically. Our numerical results show that the token-based approach yields superior performance over baseline policies, converging to the optimal policy as the maximum number of tokens increases.