Pythia: A Customizable Hardware Prefetching Framework Using Online Reinforcement Learning

Pythia is a hardware-realizable, lightweight data prefetcher that uses reinforcement learning to generate accurate, timely, and system-aware prefetch requests. Pythia formulates the prefetcher as a reinforcement learning agent. For every demand request, Pythia observes multiple different types of program context information to take a prefetch decision. For every prefetch decision, Pythia receives a numerical reward that evaluates prefetch quality under the current memory bandwidth utilization. Pythia uses this reward to reinforce the correlation between program context information and prefetch decision to generate highly accurate, timely, and system-aware prefetch requests in the future.