Accuracy and Uncertainty Analysis of Selected Methodological Approaches to Earthquake Early Warning in Europe

3 Earthquake early warning (EEW) is becoming an increasingly attractive real-time strategy for mitigating the 4 threats posed by potentially devastating incoming seismic events. As eﬀorts accelerate to develop practical 5 EEW-based solutions for earthquake-prone countries in Europe, it is important to understand and quantify 6 the level of performance that can be achieved by the underlying seismological algorithms. We conduct a 7 conceptual study on EEW performance in Europe, which explicitly focuses on the accuracy and associated 8 uncertainties of selected methodological approaches. 23 events from four diverse European testbeds are used 9 to compare the quality of EEW predictions produced by the Virtual Seismologist and PRESTo algorithms. 10 We ﬁrst examine the location and magnitude estimates of the algorithms, accounting for both bias and 11 uncertainty in the resulting predictions. We then investigate the ground-shaking prediction capabilities 12 of the source-parameter estimates, using an error metric that can explicitly capture the propagation of uncertainties in these estimates. Our work highlights the importance of accounting for EEW parameter uncertainties, which are often neglected in studies of EEW performance. Our ﬁndings can be used to inform current and future implementations of EEW systems in Europe. In addition, the evaluation metrics presented in this work can be used to determine EEW accuracy in any worldwide setting. provide the details of the algorithms to be examined in Examined Algorithms . The ﬁrst part of Results examines the quality of the algorithms’ location and magnitude estimates. The second part determines the capability of the source-parameter esti- mates to accurately predict the corresponding ground-motion amplitude, using a novel evaluation metric that captures source-parameter uncertainties and does not require knowledge on the ground-shaking threshold used for triggering alerts in the EEW system. We end with a discussion of the results in Conclusions


Introduction
We examine the theoretical performance of the Virtual Seismologist (VS) (Cua, 2005;Cua and Heaton, 118 2007;Cua et al., 2009) and the PRobabilistic and Evolutionary early warning SysTem (PRESTo) (Lancieri 119 and Zollo, 2008;Satriano et al., 2008bSatriano et al., , 2011 regional EEW algorithms across all testbeds. The similar 120 (Bayesian) structure of both algorithms enables direct comparisons to be made. 121 Virtual Seismologist (VS) operates within a Bayesian framework, in which the set of possible epicentral 122 location and magnitude values are jointly conditioned on the ground-motion amplitude measurements (as-123 sociated with P-and/or S-waves) at triggered stations and the prior PDF incorporates an existing state of 124 knowledge on relative earthquake probability. The magnitude and epicentral location estimates are subse-125 quently translated to peak ground-shaking predictions, using envelope attenuation relationships documented 126 in Cua and Heaton (2007). VS was originally part of the ShakeAlert R EEW system in California, but the 127 slow operational performance of the algorithm resulted in its removal in 2016 (Chung and Allen, 2019). A 128 version of VS is operating in Switzerland and has been tested for use in Greece, Turkey, Romania and Iceland 129 (Behr et al., 2016). 130 PRESTo estimates location using the RTLoc method proposed by Satriano et al. (2008b) and predicts 131 magnitude according to the RTMag procedure developed by Lancieri and Zollo (2008). RTLoc produces 132 multivariate normal probability density functions of hypocentral locations, based on P-wave arrival times 133 and a velocity model. RTMag uses a Bayesian framework for estimating magnitude, in which the likelihood 134 function depends on initial peak displacement measurements and RTLoc outputs. The prior PDF for a given 135 time step is the posterior distribution obtained at the previous time step, and the prior for the first time 136 step is optionally set as the Gutenberg-Richter distribution. Peak ground-motion parameters are computed 137 based on the location and magnitude estimates, using a GMM. PRESTo is currently operating in real-time 138 in Southern Italy, Turkey, Romania, and South Korea (Picozzi et al., 2015), and has also been tested for 139 application in Austria and Slovenia (Picozzi et al., 2015), as well as the Iberian Peninsula (Pazos et al., We use a common (neutral) method to determine event arrival times for both algorithms, given that P-wave 144 picking accuracy is not the focus of our evaluation. We leverage the SeisComP seismological software (see 145 Data and Resources) and specifically use the picks associated with its preferred origin for a given event.

146
This origin is automatically selected using the scevent module of the software, according to a number of 147 predefined rules (e.g., an origin is preferred to the previous one if it is computed using a greater number of 148 picks and/or produces lower travel time residuals, etc). Velocity models input to the PRESTo algorithm are 149 region-specific. The velocity models used for ITA and PYR are the same as those adopted for the generation 150 of synthetic seismograms in both testbeds (see Considered Events). We use the Tryggvason et al. (2002) 151 model for ICE and the Rigo et al. (1996) model for GRE. We use the Poisson's solid approximation to derive 152 undefined P-wave velocities from associated S-wave velocities (and vice versa), and we compute corresponding 153 3D travel-time grids using the NonLinLoc software (Lomax et al., 2000).

155
We estimate magnitudes based on seismogram data from stations that are associated with the preferred origin 156 location estimated by SeisComP. The seismograms are first processed as follows (Boore and Bommer, 2005): 1 s intervals starting from the P-wave arrival time. For PRESTo, we extract values of peak displacement 164 (P d ) in three different time windows, accounting for the vector modulus of the three-component seismogram 165 (Satriano et al., 2011). These time windows are (1) 2 s following the P-wave arrival if the P and S arrivals 166 are at least 2 s apart; (2) 4 s after the P-wave arrival if the P and S arrivals are at least 4 s apart; and (3) 2 167 s following the S-wave arrival.
Parameterisation of the Bayesian location prior for the VS algorithm depends on the number of stations 170 triggered at a given instant, and spatial constraints provided by data associated with not-yet arrived P-171 waves (Cua and Heaton, 2007). If only one station has triggered, the location is constrained to the area 172 of the associated Voronoi cell that is geometrically consistent with the surrounding non-triggered stations.

173
For two triggers, the location is assumed to lie on the hyperbola passing between both stations, in line with 174 the methodology described by Rydelek and Pujol (2004). For three triggers, the location is constrained to 175 one point, i.e. the intersection of the two hyperbolae that pass between all triggered stations. All possible 176 locations included in the prior are assigned equal weighting (i.e., a uniform distribution), and every other 177 spatial point is assigned zero probability. The P-wave velocity used to determine P-wave arrivals at stations 178 (for computing the location constraints) is taken as the average value within a 10 km depth, according to 179 the appropriate velocity model provided in Location Inputs.

186
Location and Magnitude Accuracy 187 We quantify the accuracy of the algorithms' location and magnitude components independently, to determine 188 the accuracy of the estimates for different levels of algorithmic uncertainty in the source parameters. Since 189 the quality of estimates should increase in time while the uncertainty decreases, this assessment is designed 190 to capture various accuracy/speed trade-off thresholds that may be of interest to stakeholders for guiding 191 decision-making and EEW alert issuance. Location and magnitude accuracy are quantified for each algorithm 192 using the root mean square error (RMSE) metric (Hyndman and Koehler, 2006). 193 We compare the location estimates in terms of their epicentral distance to the following selected target the mean) rather than standard deviations. This is because CV R provides a measure of relative uncertainty, 199 which is more appropriate for the large range of source-to-target distances used in the study. We specifically 200 examine the mean distance prediction of each algorithm for the first estimate that has uncertainty lower 201 than CV R = 0.3, CV R = 0.2, and CV R = 0.1 ( Figure 2).

202
It can be seen that PRESTo yields the best distance predictions across all uncertainty levels investigated 203 except CV R = 0.3. Its associated RMSE value is over 35% lower than that of VS for both CV R = 0.2 204 and CV R = 0.1, whereas the RMSE value for VS is 28% smaller in the largest uncertainty case (note that 205 no consistent performance differences are observed between real and simulated events). If we consider a both algorithms are noticeably different, with the discrepancies ranging between 29% and 64% across the 209 three cases. It is interesting to note that the VS RMSE values for epicentral distance increase as uncertainty 210 decreases, which is the opposite of what is intuitively expected (Cochran et al., 2018). This observation is 211 due to the effect of the algorithm's Bayesian prior, which significantly narrows the range of location estimates 212 (and therefore their uncertainty) after only two P-wave arrivals, thereby preventing any significant further 213 improvements that may be achieved using information from additional stations. 214 We compare the mean magnitude predictions of both algorithms, using the first estimates with standard 215 deviations (σ M ) below the following thresholds: 0.1, 0.2, and 0.3 ( Figure 3). It can be observed that the 216 results of the PRESTo algorithm are most accurate for all levels of uncertainty investigated. The PRESTo 217 RMSE value is approximately 15% lower than that of VS for σ M = 0.2, and over 20% lower in both other 218 cases (note that there are no distinct differences between the performance trends for real and simulated events). If we take a hypothetical M w 6 + RMSE normal-faulting earthquake with V s30 = 800 m/s and use and predicted ground shaking), given that site class does not influence the assessment of ground-motion 228 accuracy related to location and magnitude, and use the style-of-faulting information provided in Table 1. 229 We specifically focus on peak ground acceleration (PGA) predictions in this investigation.  On the other hand, an alert may be missed if this prediction is less than the true value. Our evaluation 241 offers a significant advantage over many previous studies of EEW ground-shaking or intensity accuracy (e.g., As an advancement over our companion paper, we use a version of the M D metric that can explicitly 245 account for the propagated uncertainty of the EEW source-parameter estimates in the resulting PGA CDF. 246 We use Monte Carlo sampling of the underlying distributions to capture all uncertainties, and calculate M D 247 according to the following equation: This study has conceptually examined the offline accuracy of the VS and PRESTo regional EEW algo-289 rithms, using seismic waveforms of 23 real and simulated events across four geographically disperse testbeds 290 in Europe. Our analyses have explicitly accounted for uncertainty in the algorithms' source parameter mea-291 surements, which represents a significant advancement over many previous studies of EEW performance that 292 only consider algorithmic point estimates. 293 We first assessed the algorithms' mean source-parameter estimates, which corresponded to various un-294 certainty thresholds that stakeholders may use to guide decision-making on EEW alert triggering. We found 295 that PRESTo was almost consistently the best-performing algorithm in terms of both location and mag-296 nitude estimation. PRESTo location estimates were over 35% more accurate than those of VS (except in 297 the case of relatively large source-parameter uncertainty, i.e. CV R =0.3), and its magnitude estimates were 298 approximately 15 to 20% better. We therefore conclude that PRESTo should be used for EEW purposes 299 that require mean estimates of location and magnitude, which is consistent with the recommendations of 300 our companion paper (Zuccolo et al., 2020) that compared the real-time operational performance of both 301 algorithms.

302
We also compared the capabilities of both algorithms in terms of ground-shaking (i.e., PGA) prediction, 303 using a well-known European GMM. Accuracy at this stage of EEW is crucial if alerts are issued based 304 on a given probability of exceeding a prescribed value of ground-motion amplitude or intensity. We used 305 a technique leveraged from sensitivity analysis to quantify the quality of GMM predictions for a given 306 set of location and magnitude estimates, which can also account for the propagation of their underlying 307 uncertainties.

308
We found that PRESTo was the best algorithm for ground-shaking prediction, if only point estimates  It is important to note that there are some limitations associated with this work. Firstly, the calibration 320 of phase detection and association parameters was only carried out for the events examined in this study, and 321 may not reflect the overall seismicity and network geometry of each area. Secondly, we did not explore the 322 sensitivity of the algorithms' Bayesian priors. For example, magnitude priors retrieved from regional hazard    Figure 5, considering the median prediction for a hypothetical lognormal GMM distribution with true median = 0.5g, known dispersion = 0.7, and predicted dispersion = 0.9.