Mexico temperature notebook

Kodi B. Arfer
Created 10 Sep 2018 • Last modified 16 Aug 2019

How things were

Stages:

  1. Mixed effects for grid cells with both satellite and ground data
  2. Predict temperature in cells with satellite data but no ground data, using the mixed model(s) fit at stage 1
  3. Predict temperature in cells with no satellite data using a spatial smoother

Rosenfeld et al. (2017) describes the method as applied to Israel.

Time shenanigans

Hu, Brunsell, Monaghan, Barlage, and Wilhelmi (2014): "The overpass times provided by MODIS LST product are in local solar time, which is defined as the MODIS observation time in coordinated universal time (UTC) plus longitude in degrees divided by 15 (Williamson, Hik, Gamon, Kavanaugh, & Koh, 2013)."

MODIS documentation: "Note that the Day_view_time and Night_view_time are in local solar time, which is the UTC time plus grid’s longitude in degrees / 15 degrees (in hours , +24 if local solar time < 0 or - 24 if local solar time >= 24). The data day in the name of all the daily MOD11A1 files is in UTC so the data day in local solar time at each grid may be different from the data day in UTC by one day."

Metereological missingness

Are there any days on which one of the ground-station meterological variables is missing from every station? 

sapply(
    subset(select = -date, ground[,
        by = date,
        .SDcols = c(temp.ground.vars, nontemp.ground.vars),
        lapply(.SD, function(v) all(is.na(v)))]),
    any)
  x
ground.temp.lo FALSE
ground.temp.mean FALSE
ground.temp.hi FALSE
wind.speed.mean FALSE

Fortunately, no.

Satellite-temperature missingness

r = rbindlist(lapply(available.years, function(y)
   {d = model.dataset(y, mrow.set = "pred.area")
    cbind(year = y, d[,
        .SDcols = c("satellite.temp.day.imputed", "satellite.temp.night.imputed"),
        lapply(.SD, mean)])}))
setnames(r, c("year", "day", "night"))

rd(d = 2, as.data.frame(r))
  year day night
1 2003 0.30 0.35
2 2004 0.31 0.37
3 2005 0.30 0.30
4 2006 0.31 0.36
5 2007 0.28 0.33
6 2008 0.29 0.30
7 2009 0.26 0.30
8 2010 0.32 0.33
9 2011 0.23 0.27
10 2012 0.31 0.35
11 2013 0.32 0.37
12 2014 0.32 0.35
13 2015 0.34 0.37
14 2016 0.34 0.36
15 2017 0.27 0.28
16 2018 0.29 0.34

Cross-validation results

sr = summarize.cv.results(multi.run.cv(available.years))

Here are RMSE and R2 by year and DV, as well as the proportion of daily Moran's I statistics of the signed error that are significant at α = .05. N and stn denote the number of observations and stations in the megalopolis, for which predictions were made; stations in the larger region, which were only used for training, aren't counted. sd.s and rmse.s are spatially weighted RMSEs, in which each day and each sixteenth of a lon–lat cell are given total weight 1. 

library(data.table)
as.data.frame(rd(d = 2, sr$overall))
  year dv N stn sd rmse R2 sd.s rmse.s R2.spatial R2.temporal Moran ps < .05
1 2003 hi 8498 29 3.43 1.20 0.88 3.69 1.63 0.83 0.89 0.25
2 2003 lo 8498 29 3.62 1.71 0.78 4.02 2.47 0.62 0.88 0.32
3 2003 mean 8498 29 3.00 0.87 0.92 3.22 1.19 0.33 0.94 0.40
4 2004 hi 9144 30 3.09 1.15 0.86 3.59 1.45 0.78 0.88 0.23
5 2004 lo 9144 30 3.31 1.58 0.77 3.87 2.34 0.23 0.88 0.14
6 2004 mean 9144 30 2.63 0.91 0.88 3.11 1.24 0.71 0.91 0.22
7 2005 hi 9844 33 3.58 1.33 0.86 4.09 1.74 0.82 0.90 0.08
8 2005 lo 9844 33 3.42 1.59 0.78 4.07 2.40 0.75 0.86 0.09
9 2005 mean 9844 33 3.05 1.01 0.89 3.52 1.42 0.74 0.92 0.09
10 2006 hi 9244 32 3.42 1.41 0.83 3.89 1.77 0.74 0.87 0.02
11 2006 lo 9244 32 3.54 1.56 0.81 4.13 2.20 0.61 0.87 0.04
12 2006 mean 9244 32 2.88 1.07 0.86 3.38 1.41 0.83 0.88 0.05
13 2007 hi 8364 34 3.29 1.28 0.85 3.78 1.63 0.77 0.86 0.15
14 2007 lo 8364 34 3.26 1.52 0.78 3.83 2.04 0.63 0.83 0.13
15 2007 mean 8364 34 2.73 0.95 0.88 3.24 1.20 0.81 0.89 0.23
16 2008 hi 10369 36 3.42 1.43 0.82 3.82 1.72 0.83 0.81 0.37
17 2008 lo 10369 36 3.94 1.91 0.76 4.60 2.50 0.75 0.83 0.21
18 2008 mean 10369 36 3.22 1.26 0.85 3.70 1.56 0.86 0.86 0.31
19 2009 hi 11563 43 3.38 1.38 0.83 3.62 1.72 0.48 0.89 0.45
20 2009 lo 11563 43 3.36 1.61 0.77 4.03 2.36 0.62 0.84 0.39
21 2009 mean 11563 43 2.87 1.12 0.85 3.24 1.55 0.74 0.91 0.52
22 2010 hi 14339 53 5.06 1.44 0.92 7.17 1.95 0.94 0.90 0.18
23 2010 lo 14339 53 4.42 1.73 0.85 5.44 2.37 0.86 0.84 0.40
24 2010 mean 14339 53 4.27 1.12 0.93 5.94 1.56 0.95 0.91 0.29
25 2011 hi 14262 46 4.77 1.62 0.89 6.62 2.10 0.90 0.86 0.10
26 2011 lo 14262 46 4.21 1.72 0.83 5.18 2.29 0.83 0.82 0.15
27 2011 mean 14262 46 4.10 1.19 0.92 5.52 1.58 0.94 0.88 0.20
28 2012 hi 15795 54 4.51 1.49 0.89 6.34 1.90 0.90 0.85 0.22
29 2012 lo 15795 54 3.90 1.72 0.81 5.04 2.29 0.85 0.75 0.36
30 2012 mean 15795 54 3.82 1.08 0.92 5.34 1.41 0.95 0.86 0.53
31 2013 hi 17870 61 4.81 1.73 0.87 6.37 2.20 0.86 0.84 0.27
32 2013 lo 17870 61 4.26 1.90 0.80 5.31 2.41 0.84 0.72 0.46
33 2013 mean 17870 61 4.15 1.16 0.92 5.45 1.41 0.96 0.84 0.60
34 2014 hi 19437 64 4.55 1.72 0.86 6.22 2.14 0.87 0.81 0.30
35 2014 lo 19437 64 4.22 1.74 0.83 5.36 2.26 0.86 0.78 0.35
36 2014 mean 19437 64 3.96 1.15 0.92 5.38 1.42 0.95 0.84 0.51
37 2015 hi 21632 76 4.54 1.77 0.85 6.36 2.17 0.85 0.79 0.21
38 2015 lo 21632 76 4.02 1.86 0.79 5.42 2.29 0.72 0.72 0.37
39 2015 mean 21632 76 3.91 1.27 0.89 5.57 1.48 0.87 0.81 0.47
40 2016 hi 25012 77 4.75 1.76 0.86 6.23 2.05 0.86 0.86 0.23
41 2016 lo 25012 77 4.30 2.05 0.77 5.42 2.18 0.78 0.77 0.64
42 2016 mean 25012 77 4.15 1.41 0.88 5.49 1.38 0.89 0.87 0.57
43 2017 hi 25077 91 4.38 1.86 0.82 6.13 2.13 0.82 0.79 0.29
44 2017 lo 25077 91 4.66 2.17 0.78 5.91 2.59 0.80 0.81 0.53
45 2017 mean 25077 91 4.15 1.54 0.86 5.69 1.60 0.87 0.85 0.55
46 2018 hi 24956 90 4.47 2.25 0.75 6.29 3.14 0.76 0.78 0.16
47 2018 lo 24956 90 4.30 2.43 0.68 5.39 2.73 0.82 0.60 0.62
48 2018 mean 24956 90 3.98 1.77 0.80 5.41 2.06 0.85 0.78 0.41

Here's plot of the above SDs and RMSEs. Each year gets a line going from the SD (top) to the RMSE (bottom). 

ggplot(transform(sr$overall,
        dv = factor(dv, levels = c("hi", "mean", "lo")))) +
    geom_linerange(aes(
        sprintf("%02d", year - 2000),
        ymin = rmse, ymax = sd)) +
    facet_grid(dv ~ ., labeller = label_both) +
    no.gridlines() +
    xlab("Year") +
    scale_y_continuous(expand = expand_scale(), name = "SD and RMSE") +
    coord_cartesian(ylim = c(0, 7))
g/rmse-by-year
rmse-by-year.png

RMSE by whether the satellite temperature was imputed (imp.d for day, imp.n for night): 

as.data.frame(rd(d = 2, sr$by.imp))
  year dv imp.d imp.n N stn sd rmse sd - rmse
1 2003 hi FALSE FALSE 4884 28 3.36 1.24 2.13
2 2003 hi FALSE TRUE 1426 28 2.82 1.05 1.77
3 2003 hi TRUE FALSE 881 29 3.08 1.17 1.91
4 2003 hi TRUE TRUE 1307 28 3.07 1.22 1.85
5 2003 lo FALSE FALSE 4884 28 3.74 1.89 1.85
6 2003 lo FALSE TRUE 1426 28 3.18 1.43 1.75
7 2003 lo TRUE FALSE 881 29 3.64 1.63 2.01
8 2003 lo TRUE TRUE 1307 28 2.71 1.32 1.39
9 2003 mean FALSE FALSE 4884 28 3.21 0.91 2.30
10 2003 mean FALSE TRUE 1426 28 2.61 0.75 1.85
11 2003 mean TRUE FALSE 881 29 2.94 0.93 2.01
12 2003 mean TRUE TRUE 1307 28 2.40 0.80 1.60
13 2004 hi FALSE FALSE 5019 30 2.75 1.09 1.66
14 2004 hi FALSE TRUE 1674 30 2.79 1.22 1.57
15 2004 hi TRUE FALSE 930 30 2.87 1.09 1.78
16 2004 hi TRUE TRUE 1521 30 3.54 1.31 2.23
17 2004 lo FALSE FALSE 5019 30 3.46 1.70 1.75
18 2004 lo FALSE TRUE 1674 30 2.70 1.39 1.32
19 2004 lo TRUE FALSE 930 30 2.94 1.66 1.28
20 2004 lo TRUE TRUE 1521 30 2.55 1.25 1.30
21 2004 mean FALSE FALSE 5019 30 2.67 0.93 1.74
22 2004 mean FALSE TRUE 1674 30 2.50 0.92 1.58
23 2004 mean TRUE FALSE 930 30 2.27 0.90 1.37
24 2004 mean TRUE TRUE 1521 30 2.65 0.84 1.81
25 2005 hi FALSE FALSE 5958 33 3.54 1.31 2.23
26 2005 hi FALSE TRUE 1227 31 3.41 1.38 2.03
27 2005 hi TRUE FALSE 1185 31 3.18 1.38 1.80
28 2005 hi TRUE TRUE 1474 32 3.00 1.37 1.63
29 2005 lo FALSE FALSE 5958 33 3.62 1.66 1.96
30 2005 lo FALSE TRUE 1227 31 2.89 1.42 1.47
31 2005 lo TRUE FALSE 1185 31 2.88 1.61 1.27
32 2005 lo TRUE TRUE 1474 32 2.43 1.40 1.03
33 2005 mean FALSE FALSE 5958 33 3.30 0.98 2.32
34 2005 mean FALSE TRUE 1227 31 2.92 1.01 1.91
35 2005 mean TRUE FALSE 1185 31 2.54 1.15 1.39
36 2005 mean TRUE TRUE 1474 32 2.25 1.04 1.21
37 2006 hi FALSE FALSE 5366 32 3.13 1.34 1.79
38 2006 hi FALSE TRUE 1468 31 3.06 1.42 1.64
39 2006 hi TRUE FALSE 822 32 3.57 1.59 1.98
40 2006 hi TRUE TRUE 1588 32 3.38 1.51 1.87
41 2006 lo FALSE FALSE 5366 32 3.70 1.66 2.04
42 2006 lo FALSE TRUE 1468 31 2.58 1.48 1.10
43 2006 lo TRUE FALSE 822 32 3.00 1.56 1.44
44 2006 lo TRUE TRUE 1588 32 2.42 1.29 1.13
45 2006 mean FALSE FALSE 5366 32 3.06 1.05 2.01
46 2006 mean FALSE TRUE 1468 31 2.54 1.13 1.41
47 2006 mean TRUE FALSE 822 32 2.63 1.06 1.57
48 2006 mean TRUE TRUE 1588 32 2.55 1.09 1.46
49 2007 hi FALSE FALSE 5099 34 2.87 1.23 1.64
50 2007 hi FALSE TRUE 1176 34 3.08 1.26 1.82
51 2007 hi TRUE FALSE 818 34 3.52 1.44 2.07
52 2007 hi TRUE TRUE 1271 34 3.66 1.40 2.26
53 2007 lo FALSE FALSE 5099 34 3.23 1.62 1.61
54 2007 lo FALSE TRUE 1176 34 2.79 1.27 1.52
55 2007 lo TRUE FALSE 818 34 2.80 1.49 1.31
56 2007 lo TRUE TRUE 1271 34 2.69 1.33 1.36
57 2007 mean FALSE FALSE 5099 34 2.71 0.97 1.74
58 2007 mean FALSE TRUE 1176 34 2.73 0.93 1.80
59 2007 mean TRUE FALSE 818 34 2.61 0.96 1.64
60 2007 mean TRUE TRUE 1271 34 2.73 0.90 1.83
61 2008 hi FALSE FALSE 6339 36 3.02 1.29 1.73
62 2008 hi FALSE TRUE 1474 36 3.21 1.60 1.61
63 2008 hi TRUE FALSE 1020 36 3.66 1.57 2.09
64 2008 hi TRUE TRUE 1536 35 3.61 1.71 1.90
65 2008 lo FALSE FALSE 6339 36 3.80 1.82 1.98
66 2008 lo FALSE TRUE 1474 36 3.87 2.42 1.45
67 2008 lo TRUE FALSE 1020 36 3.41 1.68 1.73
68 2008 lo TRUE TRUE 1536 35 3.16 1.89 1.27
69 2008 mean FALSE FALSE 6339 36 3.17 1.10 2.07
70 2008 mean FALSE TRUE 1474 36 3.41 1.68 1.73
71 2008 mean TRUE FALSE 1020 36 2.98 1.19 1.79
72 2008 mean TRUE TRUE 1536 35 3.04 1.47 1.58
73 2009 hi FALSE FALSE 7185 43 3.03 1.30 1.73
74 2009 hi FALSE TRUE 1820 43 2.83 1.46 1.37
75 2009 hi TRUE FALSE 1041 42 3.50 1.48 2.03
76 2009 hi TRUE TRUE 1517 42 3.61 1.57 2.03
77 2009 lo FALSE FALSE 7185 43 3.52 1.72 1.80
78 2009 lo FALSE TRUE 1820 43 2.88 1.52 1.36
79 2009 lo TRUE FALSE 1041 42 3.02 1.57 1.46
80 2009 lo TRUE TRUE 1517 42 2.35 1.13 1.22
81 2009 mean FALSE FALSE 7185 43 2.95 1.12 1.83
82 2009 mean FALSE TRUE 1820 43 2.59 1.16 1.43
83 2009 mean TRUE FALSE 1041 42 2.81 1.07 1.74
84 2009 mean TRUE TRUE 1517 42 2.56 1.09 1.47
85 2010 hi FALSE FALSE 8194 53 4.75 1.37 3.38
86 2010 hi FALSE TRUE 1827 53 4.06 1.42 2.63
87 2010 hi TRUE FALSE 1524 52 5.17 1.61 3.56
88 2010 hi TRUE TRUE 2794 52 5.35 1.55 3.80
89 2010 lo FALSE FALSE 8194 53 4.40 1.81 2.58
90 2010 lo FALSE TRUE 1827 53 3.33 1.52 1.82
91 2010 lo TRUE FALSE 1524 52 3.94 1.66 2.28
92 2010 lo TRUE TRUE 2794 52 4.00 1.64 2.36
93 2010 mean FALSE FALSE 8194 53 4.30 1.11 3.20
94 2010 mean FALSE TRUE 1827 53 3.50 1.08 2.42
95 2010 mean TRUE FALSE 1524 52 4.26 1.19 3.07
96 2010 mean TRUE TRUE 2794 52 4.33 1.15 3.17
97 2011 hi FALSE FALSE 9363 46 4.52 1.52 3.00
98 2011 hi FALSE TRUE 1935 46 4.19 1.61 2.58
99 2011 hi TRUE FALSE 978 46 5.86 1.84 4.02
100 2011 hi TRUE TRUE 1986 46 4.76 1.90 2.86
101 2011 lo FALSE FALSE 9363 46 4.23 1.82 2.40
102 2011 lo FALSE TRUE 1935 46 3.54 1.53 2.02
103 2011 lo TRUE FALSE 978 46 4.30 1.74 2.57
104 2011 lo TRUE TRUE 1986 46 3.43 1.37 2.06
105 2011 mean FALSE FALSE 9363 46 4.13 1.18 2.96
106 2011 mean FALSE TRUE 1935 46 3.68 1.15 2.52
107 2011 mean TRUE FALSE 978 46 4.80 1.33 3.47
108 2011 mean TRUE TRUE 1986 46 3.69 1.20 2.49
109 2012 hi FALSE FALSE 8618 54 4.10 1.43 2.67
110 2012 hi FALSE TRUE 2516 54 3.87 1.54 2.33
111 2012 hi TRUE FALSE 1931 54 5.03 1.53 3.50
112 2012 hi TRUE TRUE 2730 54 4.73 1.62 3.10
113 2012 lo FALSE FALSE 8618 54 3.88 1.90 1.98
114 2012 lo FALSE TRUE 2516 54 3.56 1.48 2.08
115 2012 lo TRUE FALSE 1931 54 3.87 1.63 2.24
116 2012 lo TRUE TRUE 2730 54 3.51 1.32 2.19
117 2012 mean FALSE FALSE 8618 54 3.76 1.10 2.66
118 2012 mean FALSE TRUE 2516 54 3.56 1.01 2.55
119 2012 mean TRUE FALSE 1931 54 4.13 1.10 3.03
120 2012 mean TRUE TRUE 2730 54 3.83 1.05 2.78
121 2013 hi FALSE FALSE 9464 61 4.23 1.66 2.57
122 2013 hi FALSE TRUE 3044 61 4.19 1.74 2.45
123 2013 hi TRUE FALSE 1925 61 5.37 1.83 3.54
124 2013 hi TRUE TRUE 3437 61 5.06 1.85 3.21
125 2013 lo FALSE FALSE 9464 61 4.27 2.09 2.18
126 2013 lo FALSE TRUE 3044 61 4.06 1.64 2.42
127 2013 lo TRUE FALSE 1925 61 4.38 1.84 2.54
128 2013 lo TRUE TRUE 3437 61 3.91 1.55 2.36
129 2013 mean FALSE FALSE 9464 61 4.02 1.18 2.84
130 2013 mean FALSE TRUE 3044 61 3.88 1.10 2.78
131 2013 mean TRUE FALSE 1925 61 4.66 1.20 3.46
132 2013 mean TRUE TRUE 3437 61 4.08 1.11 2.97
133 2014 hi FALSE FALSE 10627 64 4.33 1.67 2.66
134 2014 hi FALSE TRUE 2753 63 3.75 1.75 2.00
135 2014 hi TRUE FALSE 2386 63 4.78 1.83 2.95
136 2014 hi TRUE TRUE 3671 64 4.65 1.79 2.86
137 2014 lo FALSE FALSE 10627 64 4.30 1.91 2.38
138 2014 lo FALSE TRUE 2753 63 3.54 1.52 2.01
139 2014 lo TRUE FALSE 2386 63 4.20 1.78 2.43
140 2014 lo TRUE TRUE 3671 64 3.66 1.29 2.37
141 2014 mean FALSE FALSE 10627 64 4.05 1.17 2.88
142 2014 mean FALSE TRUE 2753 63 3.36 1.15 2.21
143 2014 mean TRUE FALSE 2386 63 4.15 1.21 2.95
144 2014 mean TRUE TRUE 3671 64 3.81 1.04 2.77
145 2015 hi FALSE FALSE 10807 76 3.98 1.74 2.25
146 2015 hi FALSE TRUE 3761 73 3.75 1.73 2.02
147 2015 hi TRUE FALSE 3168 74 4.91 1.84 3.07
148 2015 hi TRUE TRUE 3896 73 4.88 1.86 3.02
149 2015 lo FALSE FALSE 10807 76 4.07 2.07 2.01
150 2015 lo FALSE TRUE 3761 73 3.50 1.60 1.90
151 2015 lo TRUE FALSE 3168 74 4.14 1.85 2.29
152 2015 lo TRUE TRUE 3896 73 3.94 1.44 2.50
153 2015 mean FALSE FALSE 10807 76 3.75 1.34 2.41
154 2015 mean FALSE TRUE 3761 73 3.35 1.17 2.18
155 2015 mean TRUE FALSE 3168 74 4.26 1.25 3.02
156 2015 mean TRUE TRUE 3896 73 4.08 1.19 2.89
157 2016 hi FALSE FALSE 12968 77 4.38 1.70 2.68
158 2016 hi FALSE TRUE 3948 77 3.81 1.74 2.08
159 2016 hi TRUE FALSE 3445 77 5.11 1.85 3.26
160 2016 hi TRUE TRUE 4651 77 4.93 1.87 3.06
161 2016 lo FALSE FALSE 12968 77 4.38 2.17 2.21
162 2016 lo FALSE TRUE 3948 77 3.71 1.87 1.85
163 2016 lo TRUE FALSE 3445 77 4.35 2.17 2.18
164 2016 lo TRUE TRUE 4651 77 4.08 1.73 2.35
165 2016 mean FALSE FALSE 12968 77 4.08 1.41 2.68
166 2016 mean FALSE TRUE 3948 77 3.54 1.38 2.16
167 2016 mean TRUE FALSE 3445 77 4.42 1.51 2.91
168 2016 mean TRUE TRUE 4651 77 4.23 1.39 2.83
169 2017 hi FALSE FALSE 15683 91 4.17 1.79 2.38
170 2017 hi FALSE TRUE 3169 90 4.03 1.99 2.04
171 2017 hi TRUE FALSE 2672 90 5.20 2.03 3.17
172 2017 hi TRUE TRUE 3553 89 4.26 1.88 2.39
173 2017 lo FALSE FALSE 15683 91 4.65 2.33 2.32
174 2017 lo FALSE TRUE 3169 90 3.95 1.95 1.99
175 2017 lo TRUE FALSE 2672 90 4.56 2.05 2.52
176 2017 lo TRUE TRUE 3553 89 3.44 1.59 1.85
177 2017 mean FALSE FALSE 15683 91 4.14 1.54 2.60
178 2017 mean FALSE TRUE 3169 90 3.82 1.59 2.22
179 2017 mean TRUE FALSE 2672 90 4.67 1.57 3.10
180 2017 mean TRUE TRUE 3553 89 3.69 1.47 2.21
181 2018 hi FALSE FALSE 13811 90 4.19 2.22 1.97
182 2018 hi FALSE TRUE 4774 90 3.92 2.36 1.57
183 2018 hi TRUE FALSE 2632 89 4.58 2.20 2.37
184 2018 hi TRUE TRUE 3739 89 4.77 2.24 2.53
185 2018 lo FALSE FALSE 13811 90 4.32 2.46 1.87
186 2018 lo FALSE TRUE 4774 90 3.89 2.62 1.27
187 2018 lo TRUE FALSE 2632 89 4.36 2.22 2.14
188 2018 lo TRUE TRUE 3739 89 4.16 2.19 1.97
189 2018 mean FALSE FALSE 13811 90 3.92 1.72 2.20
190 2018 mean FALSE TRUE 4774 90 3.55 1.94 1.61
191 2018 mean TRUE FALSE 2632 89 4.21 1.71 2.50
192 2018 mean TRUE TRUE 3739 89 4.14 1.77 2.38

It's inconsistent whether we see greater improvement over the RMSE when both satellite temperatures are missing than when both are present: 

m = merge(
    sr$by.imp[!imp.d & !imp.n],
    sr$by.imp[imp.d & imp.n],
    by = c("year", "dv"))
m[, table(get("sd - rmse.y") > get("sd - rmse.x"))]
  count
FALSE 23
TRUE 25

Here are the RMSEs (and Moran's I p-values for the per-station mean signed error) by meteorological season: 

as.data.frame(rd(d = 2, sr$by.season))
  year dv season N stn sd rmse sd - rmse Moran p
1 2003 hi ColdDry 2896 29 3.21 1.38 1.82 0.52
2 2003 hi Rainy 4173 28 2.99 1.03 1.95 0.70
3 2003 hi WarmDry 1429 27 3.20 1.24 1.96 0.39
4 2003 lo ColdDry 2896 29 3.12 1.90 1.22 0.40
5 2003 lo Rainy 4173 28 1.99 1.46 0.53 0.03
6 2003 lo WarmDry 1429 27 3.05 1.98 1.07 0.30
7 2003 mean ColdDry 2896 29 2.63 0.99 1.63 0.50
8 2003 mean Rainy 4173 28 2.15 0.76 1.39 0.01
9 2003 mean WarmDry 1429 27 2.83 0.92 1.91 0.14
10 2004 hi ColdDry 3016 30 3.26 1.16 2.10 0.32
11 2004 hi Rainy 4730 29 2.57 1.18 1.39 0.19
12 2004 hi WarmDry 1398 25 3.06 1.02 2.04 0.79
13 2004 lo ColdDry 3016 30 3.03 1.84 1.18 0.73
14 2004 lo Rainy 4730 29 1.75 1.29 0.46 0.43
15 2004 lo WarmDry 1398 25 2.62 1.81 0.81 0.69
16 2004 mean ColdDry 3016 30 2.51 1.00 1.51 0.30
17 2004 mean Rainy 4730 29 1.83 0.84 0.99 0.29
18 2004 mean WarmDry 1398 25 2.56 0.96 1.61 0.67
19 2005 hi ColdDry 3238 32 2.96 1.26 1.71 0.55
20 2005 hi Rainy 4839 32 3.34 1.38 1.96 0.36
21 2005 hi WarmDry 1767 31 3.09 1.33 1.76 0.63
22 2005 lo ColdDry 3238 32 2.74 1.60 1.14 0.73
23 2005 lo Rainy 4839 32 2.40 1.55 0.85 0.16
24 2005 lo WarmDry 1767 31 2.98 1.69 1.29 0.30
25 2005 mean ColdDry 3238 32 2.34 0.89 1.45 0.69
26 2005 mean Rainy 4839 32 2.54 1.13 1.41 0.25
27 2005 mean WarmDry 1767 31 2.52 0.87 1.65 0.19
28 2006 hi ColdDry 3098 32 3.34 1.42 1.93 0.84
29 2006 hi Rainy 4352 31 2.84 1.42 1.42 0.50
30 2006 hi WarmDry 1794 30 2.95 1.37 1.58 0.59
31 2006 lo ColdDry 3098 32 3.29 1.66 1.63 0.82
32 2006 lo Rainy 4352 31 2.09 1.41 0.68 0.45
33 2006 lo WarmDry 1794 30 2.93 1.75 1.18 0.78
34 2006 mean ColdDry 3098 32 2.77 1.07 1.70 0.03
35 2006 mean Rainy 4352 31 2.03 1.09 0.94 0.42
36 2006 mean WarmDry 1794 30 2.49 1.01 1.48 0.81
37 2007 hi ColdDry 3078 34 3.00 1.37 1.63 0.47
38 2007 hi Rainy 3978 32 3.27 1.20 2.07 0.01
39 2007 hi WarmDry 1308 26 3.12 1.32 1.80 0.70
40 2007 lo ColdDry 3078 34 2.67 1.63 1.03 0.53
41 2007 lo Rainy 3978 32 2.58 1.40 1.18 0.00
42 2007 lo WarmDry 1308 26 3.02 1.60 1.42 0.03
43 2007 mean ColdDry 3078 34 2.31 1.02 1.28 0.03
44 2007 mean Rainy 3978 32 2.46 0.87 1.59 0.00
45 2007 mean WarmDry 1308 26 2.73 1.00 1.73 0.00
46 2008 hi ColdDry 3358 36 3.10 1.24 1.85 0.26
47 2008 hi Rainy 5213 34 3.30 1.50 1.80 0.01
48 2008 hi WarmDry 1798 33 3.37 1.57 1.80 0.21
49 2008 lo ColdDry 3358 36 3.03 1.80 1.23 0.49
50 2008 lo Rainy 5213 34 2.98 2.06 0.92 0.48
51 2008 lo WarmDry 1798 33 3.18 1.67 1.50 0.42
52 2008 mean ColdDry 3358 36 2.77 1.11 1.66 0.92
53 2008 mean Rainy 5213 34 2.77 1.41 1.36 0.18
54 2008 mean WarmDry 1798 33 3.02 1.08 1.95 0.05
55 2009 hi ColdDry 3618 43 3.45 1.42 2.03 0.80
56 2009 hi Rainy 5850 41 2.84 1.38 1.46 0.61
57 2009 hi WarmDry 2095 39 2.75 1.33 1.43 0.47
58 2009 lo ColdDry 3618 43 3.02 1.79 1.23 0.41
59 2009 lo Rainy 5850 41 1.86 1.51 0.35 0.01
60 2009 lo WarmDry 2095 39 3.02 1.52 1.50 0.59
61 2009 mean ColdDry 3618 43 2.58 1.15 1.42 0.46
62 2009 mean Rainy 5850 41 2.07 1.14 0.93 0.14
63 2009 mean WarmDry 2095 39 2.57 0.99 1.58 0.08
64 2010 hi ColdDry 4604 52 4.77 1.49 3.28 0.91
65 2010 hi Rainy 7377 49 4.70 1.39 3.31 0.99
66 2010 hi WarmDry 2358 46 4.53 1.52 3.00 0.71
67 2010 lo ColdDry 4604 52 3.71 1.98 1.73 0.85
68 2010 lo Rainy 7377 49 3.42 1.56 1.86 0.00
69 2010 lo WarmDry 2358 46 3.51 1.71 1.79 0.02
70 2010 mean ColdDry 4604 52 3.70 1.26 2.44 0.28
71 2010 mean Rainy 7377 49 3.67 1.04 2.63 0.93
72 2010 mean WarmDry 2358 46 3.85 1.07 2.78 0.65
73 2011 hi ColdDry 4654 46 4.33 1.45 2.88 0.62
74 2011 hi Rainy 7273 45 4.74 1.72 3.02 0.35
75 2011 hi WarmDry 2335 41 4.66 1.60 3.07 0.59
76 2011 lo ColdDry 4654 46 3.66 1.88 1.77 0.84
77 2011 lo Rainy 7273 45 3.70 1.60 2.10 0.37
78 2011 lo WarmDry 2335 41 4.08 1.77 2.31 0.87
79 2011 mean ColdDry 4654 46 3.67 1.17 2.49 0.69
80 2011 mean Rainy 7273 45 3.85 1.22 2.63 0.24
81 2011 mean WarmDry 2335 41 4.20 1.14 3.06 0.81
82 2012 hi ColdDry 5039 53 4.37 1.52 2.85 0.87
83 2012 hi Rainy 7916 51 4.28 1.52 2.76 0.08
84 2012 hi WarmDry 2840 50 4.16 1.39 2.77 0.41
85 2012 lo ColdDry 5039 53 3.63 1.94 1.69 0.91
86 2012 lo Rainy 7916 51 3.18 1.54 1.64 0.05
87 2012 lo WarmDry 2840 50 3.62 1.75 1.86 0.62
88 2012 mean ColdDry 5039 53 3.66 1.19 2.47 0.60
89 2012 mean Rainy 7916 51 3.38 1.03 2.35 0.00
90 2012 mean WarmDry 2840 50 3.62 0.99 2.63 0.08
91 2013 hi ColdDry 6052 59 4.49 1.69 2.81 0.12
92 2013 hi Rainy 8923 61 4.65 1.74 2.91 0.17
93 2013 hi WarmDry 2895 54 5.39 1.80 3.59 0.39
94 2013 lo ColdDry 6052 59 3.83 2.03 1.80 0.94
95 2013 lo Rainy 8923 61 3.58 1.64 1.94 0.15
96 2013 lo WarmDry 2895 54 4.89 2.31 2.58 0.94
97 2013 mean ColdDry 6052 59 3.81 1.22 2.58 0.16
98 2013 mean Rainy 8923 61 3.81 1.08 2.73 0.01
99 2013 mean WarmDry 2895 54 4.92 1.26 3.66 0.83
100 2014 hi ColdDry 6463 64 4.49 1.65 2.84 0.29
101 2014 hi Rainy 9709 61 4.18 1.76 2.42 0.63
102 2014 hi WarmDry 3265 60 4.55 1.75 2.80 0.14
103 2014 lo ColdDry 6463 64 4.03 1.96 2.08 0.79
104 2014 lo Rainy 9709 61 3.45 1.56 1.89 0.34
105 2014 lo WarmDry 3265 60 4.02 1.80 2.22 0.89
106 2014 mean ColdDry 6463 64 3.91 1.25 2.66 0.72
107 2014 mean Rainy 9709 61 3.45 1.10 2.34 0.45
108 2014 mean WarmDry 3265 60 4.07 1.06 3.01 0.00
109 2015 hi ColdDry 6917 75 4.36 1.76 2.59 0.91
110 2015 hi Rainy 11271 69 4.35 1.72 2.63 0.32
111 2015 hi WarmDry 3444 66 5.02 1.96 3.06 0.95
112 2015 lo ColdDry 6917 75 3.96 2.08 1.88 0.15
113 2015 lo Rainy 11271 69 3.36 1.55 1.80 0.48
114 2015 lo WarmDry 3444 66 4.19 2.25 1.95 0.61
115 2015 mean ColdDry 6917 75 3.80 1.32 2.47 0.12
116 2015 mean Rainy 11271 69 3.51 1.13 2.38 0.34
117 2015 mean WarmDry 3444 66 4.43 1.59 2.84 0.37
118 2016 hi ColdDry 8192 77 4.45 1.67 2.78 0.85
119 2016 hi Rainy 12577 77 4.32 1.81 2.51 0.64
120 2016 hi WarmDry 4243 73 5.45 1.79 3.66 0.43
121 2016 lo ColdDry 8192 77 4.11 2.31 1.81 0.25
122 2016 lo Rainy 12577 77 3.49 1.76 1.73 0.00
123 2016 lo WarmDry 4243 73 4.32 2.31 2.01 0.01
124 2016 mean ColdDry 8192 77 3.79 1.43 2.36 0.11
125 2016 mean Rainy 12577 77 3.68 1.38 2.30 0.07
126 2016 mean WarmDry 4243 73 4.56 1.49 3.08 0.00
127 2017 hi ColdDry 8385 90 3.96 1.69 2.27 0.71
128 2017 hi Rainy 12334 89 4.39 1.93 2.46 0.44
129 2017 hi WarmDry 4358 76 4.60 1.94 2.66 0.53
130 2017 lo ColdDry 8385 90 4.23 2.45 1.78 0.98
131 2017 lo Rainy 12334 89 3.53 1.87 1.66 0.18
132 2017 lo WarmDry 4358 76 4.28 2.36 1.92 0.34
133 2017 mean ColdDry 8385 90 3.81 1.51 2.29 0.45
134 2017 mean Rainy 12334 89 3.72 1.54 2.19 0.20
135 2017 mean WarmDry 4358 76 4.28 1.60 2.68 0.28
136 2018 hi ColdDry 7394 87 4.01 2.00 2.01 0.95
137 2018 hi Rainy 13208 89 4.21 2.32 1.89 0.61
138 2018 hi WarmDry 4354 79 4.42 2.40 2.02 0.60
139 2018 lo ColdDry 7394 87 4.14 2.34 1.80 0.88
140 2018 lo Rainy 13208 89 3.54 2.52 1.03 0.00
141 2018 lo WarmDry 4354 79 3.99 2.29 1.70 0.75
142 2018 mean ColdDry 7394 87 3.68 1.54 2.14 0.23
143 2018 mean Rainy 13208 89 3.49 1.90 1.59 0.19
144 2018 mean WarmDry 4354 79 3.90 1.72 2.18 0.22

And a plot like the previous plot, but broken down by season: 

ggplot(transform(sr$by.season,
        dv = factor(dv, levels = c("hi", "mean", "lo")))) +
    geom_linerange(aes(season, ymin = rmse, ymax = sd, color = season)) +
    facet_grid(dv ~ sprintf("%02d", year - 2000)) +
    no.gridlines() +
    scale_y_continuous(expand = expand_scale(), name = "SD and RMSE") +
    coord_cartesian(ylim = c(0, 7)) +
    theme(axis.text.x = element_text(angle = -90))
g/rmse-by-season
rmse-by-season.png 
as.data.frame(rd(d = 2, sr$by.region))
  dv region N stn sd rmse sd - rmse
1 hi Cuautla 200 1 2.66 1.59 1.07
2 hi Cuernavaca 1818 7 4.52 1.96 2.56
3 hi Pachuca 132 1 4.29 3.25 1.04
4 hi Tlaxcala-Apizaco 243 1 2.75 1.44 1.31
5 hi Toluca 1174 6 6.32 2.75 3.58
6 hi Valle de México 17341 59 3.70 1.80 1.91
7 hi Puebla-Tlaxcala 4048 15 4.62 3.57 1.04
8 lo Cuautla 200 1 1.53 1.73 -0.20
9 lo Cuernavaca 1818 7 5.14 2.27 2.87
10 lo Pachuca 132 1 3.48 2.18 1.30
11 lo Tlaxcala-Apizaco 243 1 3.10 2.64 0.46
12 lo Toluca 1174 6 3.53 2.48 1.04
13 lo Valle de México 17341 59 3.67 1.97 1.70
14 lo Puebla-Tlaxcala 4048 15 4.53 3.85 0.67
15 mean Cuautla 200 1 1.65 1.10 0.55
16 mean Cuernavaca 1818 7 4.85 1.45 3.40
17 mean Pachuca 132 1 3.28 1.15 2.12
18 mean Tlaxcala-Apizaco 243 1 2.26 1.24 1.02
19 mean Toluca 1174 6 4.43 1.69 2.74
20 mean Valle de México 17341 59 3.29 1.56 1.73
21 mean Puebla-Tlaxcala 4048 15 3.67 2.65 1.02

These by-region results are only for 2018. 

as.data.frame(rd(d = 2, sr$by.network))
  dv network N stn sd rmse sd - rmse
1 hi emas 2773 13 7.38 2.20 5.18
2 hi esimes 859 4 3.29 1.88 1.40
3 hi simat 8037 26 3.32 1.45 1.87
4 hi unam 3462 13 2.81 0.97 1.84
5 hi wunderground 9825 34 4.36 3.01 1.34
6 lo emas 2773 13 5.74 2.03 3.71
7 lo esimes 859 4 3.73 2.58 1.15
8 lo simat 8037 26 3.60 1.76 1.84
9 lo unam 3462 13 2.76 1.10 1.66
10 lo wunderground 9825 34 4.33 3.20 1.13
11 mean emas 2773 13 6.48 1.53 4.96
12 mean esimes 859 4 2.92 1.21 1.71
13 mean simat 8037 26 2.94 1.13 1.81
14 mean unam 3462 13 2.47 0.80 1.67
15 mean wunderground 9825 34 3.67 2.43 1.23

These by-network results are only for 2018.

New predictions

# d = predict.temps("~/Jdrive/PM/Just_Lab/projects/PROGRESS_physical_activity/data/intermediate/allvar_aug8.rds")

area.map()
g/area-map
area-map.png

Above is the study area, the prediction area (divided into metropolitan areas), and the stations. 

mexico.context.map()
g/mexico-context-map
mexico-context-map.png

Above is a map of Mexico with the study area highlighted. 

temp.quantiles.map(2018L)
g/map-temp-quantiles
map-temp-quantiles.png

Above are the .95 quantiles of the lows and highs, respectively, in 2018. 

change.map(2003 : 2005, 2016 : 2018)
g/map-change
map-change.png

Above is the change in mean temperatures between the two selected three-year periods. 

area.map(years = 2003 : 2005)
g/map-stations-early
map-stations-early.png

But we can see from this map of the stations in the earlier period that there were no stations in those regions that supposedly cooled so much. 

pop.map("POB65_MAS")
g/map-population
map-population.png

Above is the gridded population density in 2010 for the whole area (counting only people ages 65 and up). 

pop.map("POB65_MAS", thresholds.tempC = c(5, 30))
g/map-extreme-person-days
map-extreme-person-days.png

Above is the person-days of exposure to extreme lows or highs, respectively, in 2010. The total exposure, summed across all pixels, is:

  kind total person-days
1 ≤ 5 °C 50,820,307
2 ≥ 30 °C 23,454,639

References

Hu, L., Brunsell, N. A., Monaghan, A. J., Barlage, M., & Wilhelmi, O. V. (2014). How can we use MODIS land surface temperature to validate long-term urban model simulations? Journal of Geophysical Research, 119(6), 3185–3201. doi:10.1002/2013JD021101

Rosenfeld, A., Dorman, M., Schwartz, J., Novack, V., Just, A. C., & Kloog, I. (2017). Estimating daily minimum, maximum, and mean near surface air temperature using hybrid satellite models across Israel. Environmental Research, 159, 297–312. doi:10.1016/j.envres.2017.08.017

Williamson, S. N., Hik, D. S., Gamon, J. A., Kavanaugh, J. L., & Koh, S. (2013). Evaluating cloud contamination in clear-sky MODIS Terra daytime land surface temperatures using ground-based meteorology station observations. Journal of Climate, 26(5), 1551–1560. doi:10.1175/JCLI-D-12-00250.1