Published September 1, 2025 | Version v1

Reinforcement Learning for Dynamic Pump Scheduling under Demand Uncertainty

  • 1. ROR icon Bielefeld University
  • 2. ROR icon KWR Water Research Institute
  • 3. ROR icon University of Exeter

Description

Reliable and cost-efficient scheduling of pumps is an important task in the daily operations of urban water distribution networks (WDNs). In this work, we address the scheduling of variable-speed pumps using Reinforcement Learning (RL), which allows network controls to adapt to changes in demand in real-time after a data-driven training phase. Previous contributions have shown the general suitability of RL for control tasks in WDNs [1], [2]. However, most of them assume deterministically known demand patterns (cf. [1]) or consider uncertainty only for valve control (cf. [2]). As RL algorithms can handle uncertain environments, we explore their potential for dynamic scheduling of the network’s pumps under uncertain demand patterns. Our optimisation goal is to train a policy that complies with upper and lower pressure bounds at all nodes in the network while minimising the cost of pumping. To this end, we make use of the Soft Actor-Critic algorithm (SAC) [3]. Data for training and testing is collected using the EPANET simulator for two benchmark networks (Net1 and Anytown) with uncertainties applied to various network parameters. In all setups, the controller is trained without nodal demand information. Our study shows promising results for a pump scheduler that can reduce energy cost by a significant amount while complying with pressure bounds even for unseen scenarios.

Files

A54.pdf

Files (310.4 kB)

Name Size
md5:c475834e9c997ebc4501e071925aac5b
310.4 kB Preview Download

Additional details

Funding

European Commission
Water-Futures - Smart Water Futures: designing the next generation of urban drinking water systems 951424

Dates

Available
2025-09-01

Software