Using multi-objective reinforcement learning to inform reservoir operations for saltwater intrusion mitigation

Reservoirs are vital infrastructure for the management of water resources, and reservoir operators must often balance multiple competing needs, including hydropower generation, water supplies, and environmental flows. The Conowingo Reservoir, the focus of this study, is contained by a large hydroelectric dam on the Lower Susquehanna River, and faces water demands from Baltimore, Maryland, local recreation, and environmental flows, among others. In recent years, saltwater intrusion from the brackish Chesapeake Bay during low freshwater-flow events has become an emerging public water supply concern for communities dependent on the Conowingo Reservoir’s releases. In this study, we first develop an updated environmental flow requirement with sub-daily temporal resolution that captures tidal salinity cycles and maintains downstream flows capable of diluting salinity at safe levels. Second, this updated flow requirement will be implemented as an objective in an evolutionary multi-objective direct policy search (EMODPS) framework that combines simulation-based optimization with reinforcement learning to identify optimal reservoir operating policies. We utilize this framework to discover adaptive operation policies that mitigate saltwater intrusion events, while still meeting existing objectives, under increasingly variable hydroclimatic conditions and rising sea levels. With this approach, we demonstrate how applying these adaptive and multi-objective policies on reservoir operations can mitigate emerging water quality concerns in coastal systems and improve their resilience to increasing climate stressors.