Codes and Data Output for "Sub-city scale hourly air quality forecasting by combining models, satellite observations, and ground measurements"

All codes are written in Python 3.8.3 CAUTION: File paths may need to be adjusted based on the directory to which this data archive is downloaded. CAUTION: Basic input data (e.g. GEOS-CF data, TROPOMI data, EPA data) are not included in this archive, and must be downloaded separately. DATA PREPROCESSING CODE These codes are used for taking data (GEOS-CF, TROPOMI, EPA, etc.) downloaded from public data sources and formatting them properly for use in the proposed methodology. Data sources are listed below. - GEOS-CF Data: https://portal.nccs.nasa.gov/datashare/gmao/geos-cf/v1/ - TROPOMI Data: https://scihub.copernicus.eu/ - EPA Data: https://aqs.epa.gov/aqsweb/airdata/download_files.html#Raw - VIIRS Data: https://ladsweb.modaps.eosdis.nasa.gov/missions-and-measurements/products/VNP46A1/ - Population Density Data: https://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-density-rev11/data-download - Roadway Density Data: https://sedac.ciesin.columbia.edu/data/set/groads-global-roads-open-access-v1 PROPOSED METHODOLOGY CODE These codes implement the proposed methodology (with or without the final updating step), as well as baselines for comparison. ANALYSIS AND VISUALIZATION CODE These codes are used to analyze the performance of the proposed and/or baseline methods, and to create the visualizations used in the paper. GENERATED DATA FILES These are the data files generated by the baseline or proposed methods presented in the paper. These are .nc NetCDF files. The general format of the filename is: AreaName Type of Model (Data Sources) CODE (prediction/truth).nc AreaName: Name of the area of interest for which the results are presented. - e.g. LasVegas, NewOrleans, NewYorkCity, SaltLakeCity, SanFrancsisco Type of Model: What type of approach is used for the forecasting. - Diurnal Model: This is the "Climatology" baseline model. - Persistance Model: This is the "Persistence" baseline model. - Prediction Model: This is the proposed methodology presented in the paper. - Prediction & Kriging Model: This is the proposed methodology presented in the paper, with the final updating step applied. Data Sources: Different types of data included in the model. - (GEOS-CF): Only EPA ground data and GEOS-CF model data are considered. - (GEOS-CF + TROPOMI): As above, but with TROPOMI satellite data also considered. - (GEOS-CF + TROPOMI + MET): As above, with GEOS-CF meteorological variables included as additional inputs to the regression function. - (GEOS-CF + TROPOMI + VPN46A1): As above, but with VIIRS night-light information instead of meteorological information used as additional inputs to the regression function. - (GEOS-CF + VPN46A1): Proposed method incorporating EPA ground data, GEOS-CF model data, and VIIRS night-light data. CODE: This sequence of letters and numbers denote different settings applied - First Character: Indicates whether patterns are extracted from the full calibration period (1) or a restricted period (2) (see 3.2). - Second Character: Indicates whether patterns are combined via addition (A) or regression (R) (see 3.3). - Third Character: Indicates the weight function used: 1=no weight, 4=time-of-day, 5=periodic, 6=decaying, 7=decaying periodic (see 3.5). - Fourth Character: Indicates whether downscaling of the model outputs is done via the nearest-gridpoint (N) or linear interpolation (L) method (see 3.1). prediction/truth: This indicated whether the dataset is the "prediction" using the method in question, or the measured ground "truth" against which these predictions are validated. For example: NewOrleans Prediction Model (GEOS-CF + TROPOMI) 2A1L (prediction).nc is a prediction by the proposed methodology for the New Orleans area, using GEOS-CF and TROPOMI data, with patterns extracted from a restricted calibration period, combined via addition, with no regression weighting and linear interpolation used for downscaling.