Kathryn,
I did talk to the Metro yesterday - no idea what they used. Maybe a few will
have read it - before copies are tossed around on the tube!
Added Geoff on this email.
Ag has answered the second question. I may come back to that after
trying to answer the first part.
There are two aspects to the WG work we're doing. The first, which I've mentioned
on a number of occasions, is to prove that the perturbation process used with the WG
works. Colin Harpham sent around a load of plots to Chris/Ag/David/Geoff just before
Christmas. I have a rough draft of a paper on this which I sent to Chris yesterday. This
involves the UKCIP08 WG, but is totally independent of the change factors David is
developing for UKCIP08. This uses some earlier HadRM3 model runs. The WG is fit to
10 grid box series across the UK and then perturbed according to the differences between
the future model integrations and the control runs. We then generate future weather and
show that its characteristics are similar to what HadRM3 got directly. This has used
the same change factors (same variables) but from a different set of RCM runs.
The whole purpose of this exercise is to show that the perturbation process works.
The only way we can test this is to use RCM model runs - because they have future
runs with a big climate change. We can't use past weather data as it doesn't have
enough of a climate change. This is validation of the perturbation process.
We can additionally validate the WG using observational data - which we've done
earlier.
Return to Q2. Ag has said how the model variants get chosen. The model variants
used have a variety of ways of being chosen. Let's say we start with the 50th percentile
for rainfall. We select all model variants between 45 and 55%. Then we want temperature
at the 90th percentile. We then do a second selection of the variants already selected
that have temperature changes between 85 and 95%. As we had initially 10,000
variants, the first selection reduced this to a 1000 (as we chose 10% of them). The
second selection reduced this to 100 (as we've again chosen only 10% of them).
Now with these 100 variants, most users will average the change factors (from David)
across these 100. These average change factors (which will approximately be
at the 50% and 90% value for precipitation and temperature respectively) get passed
to the WG. The WG then simulates 100 runs of 30 years - for the already
pre-selected location (small area) and future period.
There are obviously loads of permutations as we will be allowing users to select all
percentile levels (singly for temperature or precipitation) or jointly for both from
5 to 95 % in steps of 5.
The percentile levels can be chosen based on seasons (4) and years (1). If you
select summer say, users will also get the rest of the year - using the change factors
that
go along with those for the selected model variants.
Another possibility is to select one model variant within the chosen percentile bands
and pass these change factors to the WG.
There are other possibilities, but I think we've limited the choices to these two.
The other possibility was a variant (can't think of a better word here - but not
related to the model variants) to the first. As you have 100 chosen model variants
in this example, you could chose one at random or allow each of the 100 WG
integrations to be based on a different one of the model variants. These generated
sequences will likely have greater variability than that based on the average of the
100 or that based on the single model variant.
I think this may open up a can of worms with Ag when he reads it !
Whichever of these are chosen, the use should still run the WG for
100 30-year sequences.
I think I've made the last bit on model variant selection complicated
and haven't gone back to look at what Ag has written in the User Guidance.
It ought to tell you how the change factors that the WG needs will get selected.
Cheers
Phil
At 10:07 04/01/2008, Humphrey, Kathryn (CEOSA) wrote:
Hi Ag,
Yes that makes perfect sense in terms of selecting one/several model variant/s, thanks.
I'm still a bit confused about the utility of random sampling though as this won't give
you results for a particular probability level (will it?).  I think Phil was going to
get back to me on this as well as the change factors question.
Phil, I liked your quote in the Metro this morning!
Kathryn
___________________________________________________________________________________
From: Stephens, A (Ag) [[1]mailto:A.Stephens@rl.ac.uk]
Sent: 04 January 2008 08:56
To: Humphrey, Kathryn (CEOSA)
Cc: Phil Jones; David Sexton; C.G.Kilsby@newcastle.ac.uk
Subject: RE: Questions on the weather generator
Hi Kathryn,
I can comment on your second question. Here is my understanding:
Firstly, users must run a minimum of 100 WG runs regardless of which ones they run. This
is to enforce the use of a "probabilistic" approach.
Selection by model variant will only make sense once a user has produced some runs.
After any run they will have access to the model variant IDs that were used. The use
case that gave rise to us including "selection by model variant ID" was as follows:
1. Person X does some WG runs (sampling by whatever method she chooses).
2. She uses/analyses a set of runs to produce some interesting results.
3. She is keen to do more/different analyses using the model variants that represented
that part of parameter space.
4. She has the list of model variant IDs so she can publish these so that others can use
them or she can re-use them herself in other experiments.
5. Person Y can read about what Person X did and re-produce exactly her results, or use
the same set of interesting model variants for some other experiments.
Does that make sense?
Cheers,
Ag
___________________________________________________________________________________
From: Humphrey, Kathryn (CEOSA) [[2]mailto:kathryn.humphrey@DEFRA.GSI.GOV.UK]
Sent: 03 January 2008 16:58
To: Stephens, A (Ag)
Subject: FW: Questions on the weather generator
______________________________________________
From:  Humphrey, Kathryn (CEOSA)
Sent:  03 January 2008 16:55
To:    'Phil Jones'; 'Chris Kilsby'; 'Stephens, Ag'
Subject:       Questions on the weather generator
Phil/Chris/Ag,
I'm putting together a "quick and easy" presentation on the UKCIP08 methodology for
Defra officials to give them some idea of how it's all done so they can better
appreciate what's it's potential uses may, and may not, be.
However I'm getting stuck still on some of the WG methodology!  Can you help?  (I'm not
planning on telling them this level of detail about the WG but am just bothered by the
issues below).
I'm firstly confused about the RCM change factors; are you using these to validate the
WG runs (which I do understand) or to generate them (which I don't as I thought they
were being generated using the data in final PDFs themselves)?
And I'm still confused about the reasons for allowing users to select runs by model
variant.  I think by model variant you mean each perturbed version of HadCM3 or other
single model run or emulator result that creates a point in parameter space.  Is this
right?  If so then I understand why you can't run your WG on all model variants (too
many) so selecting a random sample is a representation of parameter space.  But my
initial understand of how the WG works is that you pick a point on the PDF (say 50th
percentile) with a given probability and run the WG for that point.  But this doesn't
make sense if you are allowing users to select random/ single model variants seasons
etc. because these won't reflect a particular percentile.   Maybe it's the case that you
don't need a particular percentile for whatever use the WG data is for, but if you don't
know, how do you know how likely your WG output is and therefore what to do with the
result in terms of planning?
Apologies for my ignorance and assistance would be gratefully received!
Kind Regards,
Kathryn
Kathryn Humphrey
Climate Change Impacts and Adaptation Team, Defra
Zone 3F Ergon House, Horseferry Road, London, SW1P 3JR
tel 0207 238 3362 fax 0207 238 3341
Department for Environment, Food and Rural Affairs (Defra)
This email and any attachments is intended for the named recipient only.
If you have received it in error you have no authority to use, disclose,
store or copy any of its contents and you should destroy it and inform
the sender.
Whilst this email and associated attachments will have been checked
for known viruses whilst within Defra systems we can accept no
responsibility once it has left our systems.
Communications on Defra's computer systems may be monitored and/or
recorded to secure the effective operation of the system and for other
lawful purposes.
Prof. Phil Jones
Climatic Research Unit        Telephone +44 (0) 1603 592090
School of Environmental Sciences    Fax +44 (0) 1603 507784
University of East Anglia
Norwich                          Email    p.jones@uea.ac.uk
NR4 7TJ
UK
----------------------------------------------------------------------------
References
1. mailto:A.Stephens@rl.ac.uk
2. mailto:kathryn.humphrey@DEFRA.GSI.GOV.UK


Dear folks,
I just wanted to update you on my progress in formulating a response to
the Douglass et al. paper in the International Journal of Climatology
(IJC). There have been several developments.
First, I contacted Science to gauge their level of interest in
publishing a response to Douglass et al. I thought it was worthwhile to
"test the water" before devoting a lot of time to the preparation of a
manuscript for submission to Science. I spoke with Jesse Smith, who
handles most of the climate-related papers at Science magazine.
The bottom line is that, while Science is interested in this issue
(particularly since Douglass et al. are casting doubt on the findings of
the 2005 Santer et al. Science paper), Jesse Smith thought it was highly
unlikely that Science would carry a rebuttal of work published in a
different journal (IJC). Regretfully, I agree. Our response to Douglass
et al. does not contain any fundamentally new science - although it does
contain some new and interesting work (see below).
It's an unfortunate situation. Singer is promoting the Douglass et al.
paper as a startling "new scientific evidence", which undercuts the key
conclusions of the IPCC and CCSP Reports. Christy is using the Douglass
et al. paper to argue that his UAH group is uniquely positioned to
perform "hard-nosed" and objective evaluation of model performance, and
that it's dangerous to leave model evaluation in the hands of biased
modelers. Much as I would like to see a high-profile rebuttal of
Douglass et al. in a journal like Science or Nature, it's unlikely that
either journal will publish such a rebuttal.
So what are our options? Personally, I'd vote for GRL. I think that it
is important to publish an expeditious response to the statistical flaws
in Douglass et al. In theory, GRL should be able to give us the desired
fast turnaround time. Would GRL accept our contribution, given that the
Douglass et al. paper was published in IJC? I think they would - we've
done a substantial amount of new work (see below), and can argue, with
some justification, that our contribution is more than just a rebuttal
of Douglass et al.
Why not go for publication of a response in IJC? According to Phil, this
option would probably take too long. I'd be interested to hear any other
thoughts you might have on publication options.
Now to the science (with a lower-case "s"). I'm appending three
candidate Figures for a GRL paper. The first Figure was motivated by
discussions I've had with Karl Taylor and Tom Wigley. It's an attempt to
convey the differences between our method of comparing observed and
simulated trends (panel A) and the approach used by Douglass et al.
(panel B).
In our method, we account for both statistical uncertainties in fitting
least-squares linear trends to noisy, temporally-autocorrelated data and
for the effects of internally-generated variability. As I've described
in previous emails, we compare each of the 49 simulated T2 and T2LT
trends (i.e., the same multi-model ensemble used in our 2005 Science
paper and in the 2006 CCSP Report) with observed T2 and T2LT trends
obtained from the RSS and UAH groups. Our 2-sigma confidence intervals
on the model and observed trends are estimated as in Santer et al.
(2000). [Santer, B.D., T.M.L. Wigley, J.S. Boyle, D.J. Gaffen, J.J.
Hnilo, D. Nychka, D.E. Parker, and K.E. Taylor, 2000: Statistical
significance of trends and trend differences in layer-average
atmospheric temperature time series, J. Geophys. Res., 105, 7337-7356]
The method that Santer et al. (2000) used to compute "adjusted" trend
confidence intervals accounts for the fact that, after fitting a trend
to T2 or T2LT data, the regression residuals are typically highly
autocorrelated. If this autocorrelation is not accounted for, one could
easily reach incorrect decisions on whether the trend in an individual
time series is significantly different from zero, or whether two time
series have significantly different trends. Santer et al. (2000)
accounted for temporal autocorrelation effects by estimating r{1}, the
lag-1 autocorrelation of the regression residuals, using r{1} to
calculate an effective sample size n{e}, and then using n{e} to
determine an adjusted standard error of the least-squares linear trend.
Panel A of Figure 1 shows the 2-sigma "adjusted" standard errors for
each individual trend. Models with excessively large tropical
variability (like FGOALS-g1.0 and GFDL-CM2.1) have large adjusted
standard errors. Models with coarse-resolution OGCMs and low-amplitude
ENSO variability (like the GISS-AOM) have smaller than observed adjusted
standard errors. Neglect of volcanic forcing (i.e., absence of El
Chichon and Pinatubo-induced temperature variability) can also
contribute to smaller than observed standard errors, as in
CCCma-CGCM3.1(T47).
The dark and light grey bars in Panel A show (respectively) the 1- and
2-sigma standard errors for the RSS T2LT trend. As is visually obvious,
36 of the 49 model trends are within 1 standard error of the RSS trend,
and 47 of the 49 model trends are within 2 standard errors of the RSS
trend.
I've already explained our "paired trend test" procedure for calculating
the statistical significance of the model-versus-observed trend
differences. This involves the normalized trend difference d1:
d1 = (b{O} - b{M}) / sqrt[ (s{bO})**2 + (s{bM})**2 ]
where b{O} and b{M} represent any single pair of Observed and Modeled
trends, with adjusted standard errors s{bO} and s{bM}.
Under the assumption that d1 is normally distributed, values of d1 >
+1.96 or
Attachment Converted: "c:\eudora\attach\santer_fig01.pdf"
Attachment Converted: "c:\eudora\attach\santer_fig02.pdf"
Attachment Converted: "c:\eudora\attach\santer_fig03.pdf"


Dear Ben,
Thank you for this detailed update of your work. A few thoughts for
your consideration ...
Where to submit this:  Although I understand your and Phil's
reluctance to try IJC, it seems to me that, despite the new work
presented, this is really a comment on Douglass et al. and so rightly
belongs in IJC.  If you suspect the review and publication process
there is unacceptably long, perhaps this should be confirmed by
inquiring with the editor, as a professional courtesy.  Decide in
advance what you'd consider a reasonable turn-around time, and if the
editor says it will take longer, going with another journal makes
sense.
Figures:  They look great.  As usual, you've done a super job telling
the story in pictures.  One suggestion would be to indicate in Fig. 3
which test, or trio of tests, is the most appropriate. Now it is shown
as the blue curves, but I'd suggest making these black (and the black
ones blue) and thicker than the rest.  That way those readers who just
skim the paper and look at the figures will get the message quickly.
Observations: Have you considered including results from HadAT and
RATPAC as well as RAOBCOR?  For even greater completeness, a version
of RATPAC pared down based on the results of Randel and Wu could be
added, as could Steve Sherwood's adjusted radiosonde data.  I'd
suggest adding results from these datasets to your Fig. 1, not the
planned Fig 4, which I gather is meant to show the differences in
versions of RAOBCOR and the impact of Douglass et al.'s choice to use
and early version.
With best wishes,
Dian
----- Original Message -----
From: Ben Santer
Date: Wednesday, January 9, 2008 10:52 pm
Subject: Update on response to Douglass et al.
> Dear folks,
>
> I just wanted to update you on my progress in formulating a
> response to
> the Douglass et al. paper in the International Journal of
> Climatology
> (IJC). There have been several developments.
>
> First, I contacted Science to gauge their level of interest in
> publishing a response to Douglass et al. I thought it was
> worthwhile to
> "test the water" before devoting a lot of time to the preparation
> of a
> manuscript for submission to Science. I spoke with Jesse Smith,
> who
> handles most of the climate-related papers at Science magazine.
>
> The bottom line is that, while Science is interested in this issue
> (particularly since Douglass et al. are casting doubt on the
> findings of
> the 2005 Santer et al. Science paper), Jesse Smith thought it was
> highly
> unlikely that Science would carry a rebuttal of work published in
> a
> different journal (IJC). Regretfully, I agree. Our response to
> Douglass
> et al. does not contain any fundamentally new science - although
> it does
> contain some new and interesting work (see below).
>
> It's an unfortunate situation. Singer is promoting the Douglass et
> al.
> paper as a startling "new scientific evidence", which undercuts
> the key
> conclusions of the IPCC and CCSP Reports. Christy is using the
> Douglass
> et al. paper to argue that his UAH group is uniquely positioned to
> perform "hard-nosed" and objective evaluation of model
> performance, and
> that it's dangerous to leave model evaluation in the hands of
> biased
> modelers. Much as I would like to see a high-profile rebuttal of
> Douglass et al. in a journal like Science or Nature, it's unlikely
> that
> either journal will publish such a rebuttal.
>
> So what are our options? Personally, I'd vote for GRL. I think
> that it
> is important to publish an expeditious response to the statistical
> flaws
> in Douglass et al. In theory, GRL should be able to give us the
> desired
> fast turnaround time. Would GRL accept our contribution, given
> that the
> Douglass et al. paper was published in IJC? I think they would -
> we've
> done a substantial amount of new work (see below), and can argue,
> with
> some justification, that our contribution is more than just a
> rebuttal
> of Douglass et al.
>
> Why not go for publication of a response in IJC? According to
> Phil, this
> option would probably take too long. I'd be interested to hear any
> other
> thoughts you might have on publication options.
>
> Now to the science (with a lower-case "s"). I'm appending three
> candidate Figures for a GRL paper. The first Figure was motivated
> by
> discussions I've had with Karl Taylor and Tom Wigley. It's an
> attempt to
> convey the differences between our method of comparing observed
> and
> simulated trends (panel A) and the approach used by Douglass et
> al.
> (panel B).
>
> In our method, we account for both statistical uncertainties in
> fitting
> least-squares linear trends to noisy, temporally-autocorrelated
> data and
> for the effects of internally-generated variability. As I've
> described
> in previous emails, we compare each of the 49 simulated T2 and
> T2LT
> trends (i.e., the same multi-model ensemble used in our 2005
> Science
> paper and in the 2006 CCSP Report) with observed T2 and T2LT
> trends
> obtained from the RSS and UAH groups. Our 2-sigma confidence
> intervals
> on the model and observed trends are estimated as in Santer et al.
> (2000). [Santer, B.D., T.M.L. Wigley, J.S. Boyle, D.J. Gaffen,
> J.J.
> Hnilo, D. Nychka, D.E. Parker, and K.E. Taylor, 2000: Statistical
> significance of trends and trend differences in layer-average
> atmospheric temperature time series, J. Geophys. Res., 105, 7337-
7356]
>
> The method that Santer et al. (2000) used to compute "adjusted"
> trend
> confidence intervals accounts for the fact that, after fitting a
> trend
> to T2 or T2LT data, the regression residuals are typically highly
> autocorrelated. If this autocorrelation is not accounted for, one
> could
> easily reach incorrect decisions on whether the trend in an
> individual
> time series is significantly different from zero, or whether two
> time
> series have significantly different trends. Santer et al. (2000)
> accounted for temporal autocorrelation effects by estimating r{1},
> the
> lag-1 autocorrelation of the regression residuals, using r{1} to
> calculate an effective sample size n{e}, and then using n{e} to
> determine an adjusted standard error of the least-squares linear
> trend.
> Panel A of Figure 1 shows the 2-sigma "adjusted" standard errors
> for
> each individual trend. Models with excessively large tropical
> variability (like FGOALS-g1.0 and GFDL-CM2.1) have large adjusted
> standard errors. Models with coarse-resolution OGCMs and low-
> amplitude
> ENSO variability (like the GISS-AOM) have smaller than observed
> adjusted
> standard errors. Neglect of volcanic forcing (i.e., absence of El
> Chichon and Pinatubo-induced temperature variability) can also
> contribute to smaller than observed standard errors, as in
> CCCma-CGCM3.1(T47).
>
> The dark and light grey bars in Panel A show (respectively) the 1-
> and
> 2-sigma standard errors for the RSS T2LT trend. As is visually
> obvious,
> 36 of the 49 model trends are within 1 standard error of the RSS
> trend,
> and 47 of the 49 model trends are within 2 standard errors of the
> RSS
> trend.
>
> I've already explained our "paired trend test" procedure for
> calculating
> the statistical significance of the model-versus-observed trend
> differences. This involves the normalized trend difference d1:
>
> d1 = (b{O} - b{M}) / sqrt[ (s{bO})**2 + (s{bM})**2 ]
>
> where b{O} and b{M} represent any single pair of Observed and
> Modeled
> trends, with adjusted standard errors s{bO} and s{bM}.
>
> Under the assumption that d1 is normally distributed, values of d1
> >
> +1.96 or
that
> are significant at some stipulated significance level, and one can
> easily calculate a p-value for each value of d1. These p-values
> for the
> 98 pairs of trend tests (49 involving UAH data and 49 involving
> RSS
> data) are what we use for determining the total number of "hits",
> or
> rejections of the null hypothesis of no significant difference
> between
> modeled and observed trends. I note that each test is two-tailed,
> since
> we have no information a priori about the "direction" of the model
> trend
> (i.e., whether we expect the simulated trend to be significantly
> larger
> or smaller than observed).
>
> REJECTION RATES FOR "PAIRED TREND TESTS, OBS-vs-MODEL
> Stipulated sign. level     No. of tests       T2 "Hits"     T2LT
> "Hits"  5%                        49 x 2    (98)     2  (2.04%)
> 1  (1.02%)
> 10%                        49 x 2    (98)     4  (4.08%)     2
> (2.04%)15%                        49 x 2    (98)     7  (7.14%)
> 5  (5.10%)
>
> Now consider Panel B of Figure 1. It helps to clarify the
> differences
> between the Douglass et al. comparison of model and observed
> trends and
> our own comparison. The black horizontal line ("Multi-model mean
> trend")
> is the T2LT trend in the 19-model ensemble, calculated from model
> ensemble mean trends (the colored symbols). Douglass et al.'s
> "consistency criterion", sigma{SE}, is given by:
>
> sigma{SE} = sigma / sqrt(N - 1)
>
> where sigma is the standard deviation of the 19 ensemble-mean
> trends,
> and N is 19. The orange and yellow envelopes denote the 1- and
> 2-sigma{SE} regions.
>
> Douglass et al. use sigma{SE} to decide whether the multi-model
> mean
> trend is consistent with either of the observed trends. They
> conclude
> that the RSS and UAH trends lie outside of the yellow envelope
> (the
> 2-sigma{SE} region), and interpret this as evidence of a
> fundamental
> inconsistency between modeled and observed trends. As noted
> previously,
>  Douglass et al. obtain this result because they fail to account
> for
> statistical uncertainty in the estimation of the RSS and UAH
> trends.
> They ignore the statistical error bars on the RSS and UAH trends
> (which
> are shown in Panel A). As is clear from Panel A, the statistical
> error
> bars on the RSS and UAH trends overlap with the Douglass et al.
> 2-sigma{SE} region. Had Douglass et al. accounted for statistical
> uncertainty in estimation of the observed trends, they would have
> been
> unable to conclude that all "UAH and RSS satellite trends are
> inconsistent with model trends".
>
> The second Figure plots values of our test statistic (d1) for the
> "paired trend test". The grey histogram is based on the values of
> d1 for
> the 49 tests involving the RSS T2LT trend and the simulated T2LT
> trends
> from 20c3m runs. The green histogram is for the 49 paired trend
> tests
> involving model 20c3m data and the UAH T2LT trend. Note that the
> d1
> distribution obtained with the UAH data is negatively skewed. This
> is
> because the numerator of the d1 test statistic is b{O} - b{M}, and
> the
> UAH tropical T2LT trend over 1979-1999 is smaller than most of the
> model
> trends (see Figure 1, panel A).
>
> The colored dots are values of the d1 test statistic for what I
> referred
> to previously as "TYPE2" tests. These tests are limited to the M
> models
> with multiple realizations of the 20c3m experiment. Here, M = 11.
> For
> each of these M models, I performed paired trend tests for all C
> unique
> combinations of trends pairs. For example, for a model with 5
> realizations of the 20c3m experiment, like GISS-EH, C = 10. The
> significance of trend differences is solely a function of "within-
> model"
> effects (i.e., is related to the different manifestations of
> natural
> internal variability superimposed on the underlying forced
> response).
> There are a total of 62 paired trend tests. Note that the
> separation of
> the colored symbols on the y-axis is for visual display purposes
> only,
> and facilitates the identification of results for individual models.
>
> The clear message from Figure 2 is that the values of d1 arising
> from
> internal variability alone are typically as large as the d1 values
> obtained by testing model trends against observational data. The
> two
> negative "outlier" values of d1 for the model-versus-observed
> trend
> tests involve the large positive trend in CCCma-CGCM3.1(T47). If
> you
> have keen eagle eyes, you'll note that the distribution of colored
> symbols is slightly skewed to the negative side. If you look at
> Panel A
> of Figure 1, you'll see that this skewness arises from the
> relatively
> small ensemble sizes. Consider results for the 5-member ensemble
> of
> 20c3m trends from the MRI-CGCM2.3.2. The trend in realization 1 is
> close
> to zero; trends in realizations 2, 3, 4, and 5 are large,
> positive, and
> vary between 0.27 to 0.37 degrees C/decade. So d1 is markedly
> negative
> for tests involving realization 1 versus realizations 2, 3, 4, and
> 5. If
> we showed non-unique combinations of trend pairs (e.g.,
> realization 2
> versus realization 1, as well as 1 versus 2), the distribution of
> colored symbols would be symmetric. But I was concerned that we
> might be
> accused of "double counting" if we did this....
>
> The third Figure is the most interesting one. You have not seen
> this
> yet. I decided to examine how the Douglass et al. "consistency
> test"
> behaves with synthetic data. I did this as a function of sample
> size N,
> for N values ranging from 19 (the number of models we used in the
> CCSP
> report) to 100. Consider the N = 19 case first. I generated 19
> synthetic
> time series using an AR-1 model of the form:
>
>               xt(i) = a1 * (xt(i-1) - am) + zt(i) + am
>
> where a1 is the coefficient of the AR-1 model, zt(i) is a
> randomly-generated noise term, and am is a mean (set to zero
> here).
> Here, I set a1 to 0.86, close to the lag-1 autocorrelation of the
> UAH
> T2LT anomaly data. The other free parameter is a scaling term
> which
> controls the amplitude of zt(i). I chose this scaling term to
> yield a
> temporal standard deviation of xt(i) that was close to the
> temporal
> standard deviation of the monthly-mean UAH T2LT anomaly data. The
> synthetic time series had the same length as the observational and
> model
> data (252 months), and monthly-mean anomalies were calculated in
> the
> same way as we did for observations and models.
>
> For each of these 19 synthetic time series, I first calculated
> least-squares linear trends and adjusted standard errors, and then
> performed the "paired trends". The test involves all 171 unique
> pairs of
> trends: b{1} versus b{2}, b{1} versus b{3},... b{1} versus b{19},
> b{2}
> versus b{3}, etc. I then calculate the rejection rates of the null
> hypothesis of "no significant difference in trend", for stipulated
> significance levels of 5%, 10%, and 20%. This procedure is
> repeated 1000
> times, with 1000 different realizations of 19 synthetic time
> series. We
> can therefore build up a distribution of rejection rates for N =
> 19, and
> then do the same for N = 20, etc.
>
> The "paired trend" results are plotted as the blue lines in Figure
> 3.
> Encouragingly, the percentage rejections of the null hypothesis
> are
> close to the theoretical expectations. The 5% significance tests
> yield a
> rejection rate of a little over 6%; 10% tests have a rejection
> rate of
> over 11%, and 20% tests have a rejection rate of 21%. I'm not
> quite sure
> why this slight positive bias arises. This bias does show some
> small
> sensitivity (1-2%) to choice of the a1 parameter and the scaling
> term.
> Different choices of these parameters can give rejection rates
> that are
> closer to the theoretical expectation. But my parameter choices
> for the
> AR-1 model were guided by the goal of generating synthetic data
> with
> roughly the same autocorrelation and variance properties as the
> UAH
> data, and not by a desire to get as close as I possibly could to
> the
> theoretical rejection rates.
>
> So why is there a small positive bias in the empirically-
> determined
> rejection rates? Perhaps Francis can provide us with some guidance
> here.
> Karl believes that the answer may be partly linked to the skewness
> of
> the empirically-determined rejection rate distributions. For
> example,
> for the N = 19 case, and for 5% tests, values of rejection rates
> in the
> 1000-member distribution range from a minimum of 0 to a maximum of
> 24%,
> with a mean value of 6.7% and a median of 6.4%. Clearly, the
> minimum
> value is bounded by zero, but the maximum is not bounded, and in
> rare
> cases, rejection rates can be quite large, and influences the
> mean. This
> inherent skewness must make some contribution to the small
> positive bias
> in rejection rates in the "paired trends" test.
>
> What happens if we naively perform the paired trends test WITHOUT
> adjusting the standard errors of the trends for temporal
> autocorrelation
> effects? Results are shown by the black lines in Figure 3. If we
> ignore
> temporal autocorrelation, we get the wrong answer. Rejection rates
> for
> 5% tests are 60%!
>
> We did not publish results from any of these synthetic data
> experiments
> in our 2000 JGR paper. In retrospect, this is a bit of a shame,
> since
> Figure 3 nicely shows that the adjustment for temporal
> autocorrelation
> effects works reasonably well, while failure to adjust yields
> completely
> erroneous results.
>
> Now consider the red lines in Figure 3. These are the results of
> applying the Douglass et al. "consistency test" to synthetic data.
> Again, let's consider the N = 19 case first. I calculate the
> trends in
> all 19 synthetic time series. Let's consider the first of these 19
> time
> series as the surrogate observations. The trend in this time
> series,
> b{1}, is compared with the mean trend, b{Synth}, computed from the
> remaining 18 synthetic time series. The Douglass sigma{SE} is also
> computed from these 18 remaining trends. We then form a test
> statistic
> d2 = (b{1} - b{Synth}) / sigma{SE}, and calculate rejection rates
> for
> the null hypothesis of no significant difference between the mean
> trend
> and the trend in the surrogate observations. This procedure is
> then
> repeated with the trend in time series 2 as the surrogate
> observations,
> and b{Synth} and sigma{SE} calculated from time series 1, 3,
> 4,..19.
> This yields 19 different tests of the null hypothesis. Repeat
> 1,000
> times, and build up a distribution of rejection rates, as in the
> "paired
> trends" test.
>
> The results are truly alarming. Application of the Douglass et al.
> "consistency test" to synthetic data - data generated with the
> same
> underlying AR-1 model! - leads to rejection of the above-stated
> null
> hypothesis at least 65% of the time (for N = 19, 5% significance
> tests).
>  As expected, rejection rates for the Douglass consistency test
> rise as
> N increases. For N = 100, rejection rates for 5% tests are nearly
> 85%.
> As my colleague Jim Boyle succinctly put it when he looked at
> these
> results, "This is a pretty hard test to pass".
>
> I think this nicely illustrates the problems with the statistical
> approach used by Douglass et al. If you want to demonstrate that
> modeled
> and observed temperature trends are fundamentally inconsistent,
> you
> devise a fundamentally flawed test is very difficult to pass.
>
> I hope to have a first draft of this stuff written up by the end
> of next
> week. If Leo is agreeable, Figure 4 of this GRL paper would show
> the
> vertical profiles of tropical temperature trends in the various
> versions
> of the RAOBCORE data, plus model results.
>
> Sorry to bore you with all the gory details. But as we've seen
> from
> Douglass et al., details matter.
>
> With best regards,
>
> Ben
> -------------------------------------------------------------------
> ---------
> Benjamin D. Santer
> Program for Climate Model Diagnosis and Intercomparison
> Lawrence Livermore National Laboratory
> P.O. Box 808, Mail Stop L-103
> Livermore, CA 94550, U.S.A.
> Tel:   (925) 422-2486
> FAX:   (925) 422-7675
> email: santer1@llnl.gov
> -------------------------------------------------------------------
> ---------
>
>


Dear Phil,
If you get a chance, could you call me up at work (+1 925 423-3364) to
talk about the "IJC publication" option? I'd really like to discuss that
with you.
With best regards,
Ben
Phil Jones wrote:
>
>  Ben,
>     Almost said something about this in the main email about the diagrams!
>  Other emails and a couple of phone calls distracting  me - have to make
> sure
>  I'm sending the right email to the right list/person!
>     He's clearly biased, but he gets an audience unfortunately. There are
>  enough people out there who think we're wrong to cause me to worry at
> times.
>     I'd like the world to warm up quicker, but if it did, I know that
> the sensitivity
>  is much higher and humanity would be in a real mess!
>
>      I'm getting people misinterpreting my comment that went along with
>  Chris Folland's press release about the 2008 forecast. It says we're
>  warming at 0.2 degC/decade and that is exactly what we should be.
>  The individual years don't matter.
>
>    CA are now to send out FOIA requests for the Review Editor comments
>  on the AR4 Chapters.  For some reason they think they exist!
>
>  Cheers
>  Phil
>
>
> At 16:52 09/01/2008, you wrote:
>> Dear Phil,
>>
>> I can't believe John is now arguing that he's the only guy who can
>> provide unbiased assessments of model performance. After all the
>> mistakes he's made with MSU, and after the Douglass et al. fiasco, he
>> should have acquired a little humility. But I guess "humility" isn't
>> in his dictionary...
>>
>> With best regards,
>>
>> Ben
>> Phil Jones wrote:
>>>  Ben,
>>>     I'll give up on trying to catch him on the road to Damascus -
>>>  he's beyond redemption.
>>>     Glad to see that someone's rejected something he's written.
>>>  Jim Hack's good, so I'm confident he won't be fooled.
>>>  Cheers
>>>  Phil
>>>
>>> At 17:28 07/01/2008, you wrote:
>>>> Dear Phil,
>>>>
>>>> More Christy stuff... The guy is just incredible...
>>>>
>>>> With best regards,
>>>>
>>>> Ben
>>>> ----------------------------------------------------------------------------
>>>>
>>>> Benjamin D. Santer
>>>> Program for Climate Model Diagnosis and Intercomparison
>>>> Lawrence Livermore National Laboratory
>>>> P.O. Box 808, Mail Stop L-103
>>>> Livermore, CA 94550, U.S.A.
>>>> Tel:   (925) 422-2486
>>>> FAX:   (925) 422-7675
>>>> email: santer1@llnl.gov
>>>> ----------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>> X-Account-Key: account1
>>>> Return-Path:
>>>> Received: from mail-2.llnl.gov ([unix socket])
>>>>         by mail-2.llnl.gov (Cyrus v2.2.12) with LMTPA;
>>>>         Mon, 07 Jan 2008 09:00:41 -0800
>>>> Received: from nspiron-2.llnl.gov (nspiron-2.llnl.gov [128.115.41.82])
>>>>         by mail-2.llnl.gov (8.13.1/8.12.3/LLNL evision: 1.6 $) with
>>>> ESMTP id m07H0edp031523;
>>>>         Mon, 7 Jan 2008 09:00:40 -0800
>>>> X-Attachments: None
>>>> X-IronPort-AV: E=McAfee;i="5100,188,5200"; a="5944377"
>>>> X-IronPort-AV: E=Sophos;i="4.24,254,1196668800";
>>>>    d="scan'208";a="5944377"
>>>> Received: from dione.llnl.gov (HELO [128.115.57.29]) ([128.115.57.29])
>>>>   by nspiron-2.llnl.gov with ESMTP; 07 Jan 2008 09:00:40 -0800
>>>> Message-ID:
>>>> Date: Mon, 07 Jan 2008 09:00:40 -0800
>>>> From: Ben Santer
>>>> Reply-To: santer1@llnl.gov
>>>> Organization: LLNL
>>>> User-Agent: Thunderbird 1.5.0.12 (X11/20070529)
>>>> MIME-Version: 1.0
>>>> To: "Hack, James J."
>>>> Subject: Re: John Christy's latest ideas
>>>> References:
>>>>
>>>> In-Reply-To:
>>>>
>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>> Content-Transfer-Encoding: 7bit
>>>>
>>>> Dear Jim,
>>>>
>>>> I'm well aware of this paper, and am currently preparing a reply
>>>> (together with many others who were involved in the first CCSP
>>>> report). To put it bluntly, the Douglass paper is a piece of
>>>> worthless garbage. It has serious statistical flaws. Christy should
>>>> be ashamed that he's a co-author on this. His letter to Dr. Strayer
>>>> is deplorable and offensive. For over a decade, Christy has
>>>> portrayed himself as the only guy who is smart enough to develop
>>>> climate-quality data records from MSU. Recently, he's also portrayed
>>>> himself as the only guy who's smart enough to develop
>>>> climate-quality data records from radiosonde data. And now he's the
>>>> only scientist who is capable of performing "hard-nosed",
>>>> independent assessments of climate model performance.
>>>>
>>>> John Christy has made a scientific career out of being wrong. He's
>>>> not even a third-rate scientist. I'd be happy to discuss Christy's
>>>> "unique ways of validating climate models" with you.
>>>>
>>>> With best regards,
>>>>
>>>> Ben
>>>> Hack, James J. wrote:
>>>>> Dear Ben,
>>>>>
>>>>> Happy New Year.  Hope all is well.  I was wondering if you're
>>>>> familiar with the attached paper?  I thought that you had recently
>>>>> published something that concludes something quite different.  Is
>>>>> that right?  If yes, could you forward me a copy?  And, any
>>>>> comments are also welcome.
>>>>> He's coming to ORNL next week to under the premise that he has some
>>>>> unique ways to validate climate models (this time with regard to
>>>>> the lower thermodynamic structure).  I'd be happy to chat with you
>>>>> about this as well if you would like.  I'm appending what I know to
>>>>> the bottom of this note.
>>>>>
>>>>> Best regards ...
>>>>>
>>>>> Jim
>>>>>
>>>>> James J. Hack Director, National Center for Computational Sciences
>>>>> Oak Ridge National Laboratory
>>>>> One Bethel Valley Road
>>>>> P.O. Box 2008, MS-6008
>>>>> Oak Ridge, TN  37831-6008
>>>>>
>>>>> email:   jhack@ornl.gov
>>>>> voice:  865-574-6334
>>>>> fax:      865-241-9578
>>>>> cell:     865-206-9001
>>>>>
>>>>>
>>>>>>  >> -----Original Message-----
>>>>>>  >> From: John Christy [_mailto:john.christy@nsstc.uah.edu_]
>>>>>>  >> Sent: Tuesday, October 23, 2007 9:16 AM
>>>>>>  >> To: Strayer, Michael
>>>>>>  >> Cc: Salmon, Jeffrey
>>>>>>  >> Subject: Climate Model Evaluation
>>>>>>  >>
>>>>>>  >> Dr. Strayer:
>>>>>>  >>
>>>>>>  >> Jeff Salmon is aware of a project we at UAHuntsville believe is
>>>>>>  >> vital and that you may provide a way to see it accomplished.
>>>>>> As you
>>>>>>  >> know, our nation's energy and climate change policies are being
>>>>>>  >> driven by output from global climate models. However, there has
>>>>>>  >> never been a true "red team" assessment of these model
>>>>>> projections
>>>>>>  >> in the way other government programs are subjected to hard-nosed,
>>>>>>  >> independent evaluations. To date, most of the "evaluation" of
>>>>>> these
>>>>>>  >> models has been left in the hands of the climate modelers
>>>>>>  >> themselves. This has the potential of biasing the entire process.
>>>>>>  >>
>>>>>>  >> It is often a climate modeler's claim (and promoted in IPCC
>>>>>>  >> documents - see attached) that the models must be correct because
>>>>>>  >> the global surface
>>>>>>  >> temperature variations since 1850 are reproduced (somewhat) by
>>>>>> the
>>>>>>  >> models when run in hindcast mode. However, this is not a
>>>>>> scientific
>>>>>>  >> experiment for the simple reason that every climate modeler
>>>>>> saw the
>>>>>>  >> answer ahead of time. It is terribly easy to get the right answer
>>>>>>  >> for the wrong reason, especially if you already know the answer.
>>>>>>  >>
>>>>>>  >> A legitimate experiment is to test the models' output against
>>>>>>  >> variables to which modelers did not have access ... a true blind
>>>>>>  >> test of the models.
>>>>>>  >>
>>>>>>  >> I have proposed and have had rejected a model evaluation
>>>>>> project to
>>>>>>  >> DOE based on the utilization of global datasets we build here at
>>>>>>  >> UAH. We have published many of these datasets (most are
>>>>>>  >> satellite-based) which document the complexity of the climate
>>>>>>  >> system and which we think models should replicate in some way,
>>>>>> and
>>>>>>  >> to aid in model development where shortcomings are found.
>>>>>> These are
>>>>>>  >> datasets of quantities that modelers in general were not aware of
>>>>>>  >> when doing model testing. We have performed
>>>>>>  >> a few of these tests and have found models reveal serious
>>>>>>  >> shortcomings in some of the most fundamental aspects of energy
>>>>>>  >> distribution. We believe a rigorous test of climate models is in
>>>>>>  >> order as the congress starts considering energy reduction
>>>>>>  >> strategies which can have significant consequences on our
>>>>>> economy.
>>>>>>  >> Below is an abstract of a retooled proposal I am working on.
>>>>>>  >>
>>>>>>  >> If you see a possible avenue for research along these lines,
>>>>>> please
>>>>>>  >> let me know. Too, we have been considering some type of
>>>>>> partnership
>>>>>>  >> with Oakridge since the facility is nearby, and this may be a way
>>>>>>  >> to do that.
>>>>>>  >>
>>>>>>  >> John C.
>>>>>>  >>
>>>>>>  >>
>>>>>>  >>
>>>>>>  >> Understanding the vertical energy distribution of the Earth's
>>>>> atmosphere
>>>>>>  >> and its expression in global climate model simulations
>>>>>>  >>
>>>>>>  >> John R. Christy, P.I., University of Alabama in Huntsville
>>>>>>  >>
>>>>>>  >> Abstract
>>>>>>  >>
>>>>>>  >> Sets of independent observations indicate, unexpectedly, that the
>>>>>>  >> warming of the tropical atmosphere since 1978 is proceeding at a
>>>>>>  >> rate much less than that anticipated from climate model
>>>>>> simulations.
>>>>>>  >> Specifically, while the surface has warmed, the lower troposphere
>>>>>>  >> has experienced less warming. In contrast, all climate models we
>>>>>>  >> and others have examined indicate the lower tropical atmosphere
>>>>>>  >> should be warming at a rate 1.2 to 1.5 times greater than the
>>>>>>  >> surface when forced with increasing greenhouse gases within the
>>>>>>  >> context of other observed forcings (the so-called "negative lapse
>>>>>>  >> rate feedback".) We propose to diagnose this curious phenomenon
>>>>>>  >> with several satellite-based datasets to document its relation to
>>>>>>  >> other climate variables. We shall do the same for climate model
>>>>>>  >> output of the same simulated variables. This will
>>>>>>  >> enable us to propose an integrated conceptual framework of the
>>>>>>  >> phenomenon for further testing. Tied in with this research are
>>>>> potential
>>>>>>  >> answers to fundamental questions such as the following: (1) In
>>>>>>  >> response to increasing surface temperatures, is the lower
>>>>>>  >> atmosphere reconfiguring the way heat energy is transported which
>>>>>>  >> allows for an increasing amount of heat to more freely escape to
>>>>>>  >> space? (2) Could there be a natural thermostatic effect in the
>>>>>>  >> climate system which acts in a different way than parameterized
>>>>>>  >> convective-adjustment schemes dependent upon current
>>>>>> assumptions of
>>>>>>  >> heat deposition and retention? (3)
>>>>>>  >> If observed atmospheric heat retention is considerably less than
>>>>>>  >> model projections, what impact will lower retention rates have on
>>>>>>  >> anticipated increases in surface temperatures in the 21st
>>>>>> century?
>>>>>>  >>
>>>>
>>>>
>>>> --
>>>> ----------------------------------------------------------------------------
>>>>
>>>> Benjamin D. Santer
>>>> Program for Climate Model Diagnosis and Intercomparison
>>>> Lawrence Livermore National Laboratory
>>>> P.O. Box 808, Mail Stop L-103
>>>> Livermore, CA 94550, U.S.A.
>>>> Tel:   (925) 422-2486
>>>> FAX:   (925) 422-7675
>>>> email: santer1@llnl.gov
>>>> ----------------------------------------------------------------------------
>>>>
>>> Prof. Phil Jones
>>> Climatic Research Unit        Telephone +44 (0) 1603 592090
>>> School of Environmental Sciences    Fax +44 (0) 1603 507784
>>> University of East Anglia
>>> Norwich                          Email    p.jones@uea.ac.uk
>>> NR4 7TJ
>>> UK
>>> ----------------------------------------------------------------------------
>>>
>>
>>
>> --
>> ----------------------------------------------------------------------------
>>
>> Benjamin D. Santer
>> Program for Climate Model Diagnosis and Intercomparison
>> Lawrence Livermore National Laboratory
>> P.O. Box 808, Mail Stop L-103
>> Livermore, CA 94550, U.S.A.
>> Tel:   (925) 422-2486
>> FAX:   (925) 422-7675
>> email: santer1@llnl.gov
>> ----------------------------------------------------------------------------
>>
>
> Prof. Phil Jones
> Climatic Research Unit        Telephone +44 (0) 1603 592090
> School of Environmental Sciences    Fax +44 (0) 1603 507784
> University of East Anglia
> Norwich                          Email    p.jones@uea.ac.uk
> NR4 7TJ
> UK
> ----------------------------------------------------------------------------
>
--
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel:   (925) 422-2486
FAX:   (925) 422-7675
email: santer1@llnl.gov
----------------------------------------------------------------------------


Dear Tim,
Thanks very much for your email. I greatly appreciate the additional
information that you've given me. I am a bit conflicted about what we
should do.
IJC published a paper with egregious statistical errors. Douglass et al.
was essentially a commentary on work by myself and colleagues - work
that had been previously published in Science in 2005 and in Chapter 5
of the first U.S. CCSP Report in 2006. To my knowledge, none of the
authors or co-authors of the Santer et al. Science paper or of CCSP 1.1
Chapter 5 were used as reviewers of Douglass et al. I am assuming that,
when he submitted his paper to IJC, Douglass specifically requested that
certain scientists should be excluded from the review process. Such an
approach is not defensible for a paper which is largely a comment on
previously-published work.
It would be fair and reasonable to give IJC the opportunity to "set the
record straight", and correct the harm they have done by publication of
Douglass et al. I use the word "harm" advisedly. The author and
coauthors of the Douglass et al. IJC paper are using this paper to argue
that "Nature, not CO2, rules the climate", and that the findings of
Douglass et al. invalidate the "discernible human influence" conclusions
of previous national and international scientific assessments.
Quick publication of a response to Douglass et al. in IJC would go some
way towards setting the record straight. I am troubled, however, by the
very real possibility that Douglass et al. will have the last word on
this subject. In my opinion (based on many years of interaction with
these guys), neither Douglass, Christy or Singer are capable of
admitting that their paper contained serious scientific errors. Their
"last word" will be an attempt to obfuscate rather than illuminate. They
are not interested in improving our scientific understanding of the
nature and causes of recent changes in atmospheric temperature. They are
solely interested in advancing their own agendas. It is telling and
troubling that Douglass et al. ignored radiosonde data showing
substantial warming of the tropical troposphere - data that were in
accord with model results - even though such data were in their
possession. Such behaviour constitutes intellectual dishonesty. I
strongly believe that leaving these guys the last word is inherently unfair.
If IJC are interested in publishing our contribution, I believe it's
fair to ask for the following:
1) Our paper should be regarded as an independent contribution, not as a
comment on Douglass et al. This seems reasonable given i) The
substantial amount of new work that we have done; and ii) The fact that
the Douglass et al. paper was not regarded as a comment on Santer et al.
(2005), or on Chapter 5 of the 2006 CCSP Report - even though Douglass
et al. clearly WAS a comment on these two publications.
2) If IJC agrees to 1), then Douglass et al. should have the opportunity
to respond to our contribution, and we should be given the chance to
reply. Any response and reply should be published side-by-side, in the
same issue of IJC.
I'd be grateful if you and Phil could provide me with some guidance on
1) and 2), and on whether you think we should submit to IJC. Feel free
to forward my email to Glenn McGregor.
With best regards,
Ben
Tim Osborn wrote:
> At 03:52 10/01/2008, Ben Santer wrote:
>> ...Much as I would like to see a high-profile rebuttal of Douglass et
>> al. in a journal like Science or Nature, it's unlikely that either
>> journal will publish such a rebuttal.
>>
>> So what are our options? Personally, I'd vote for GRL. I think that it
>> is important to publish an expeditious response to the statistical
>> flaws in Douglass et al. In theory, GRL should be able to give us the
>> desired fast turnaround time...
>>
>> Why not go for publication of a response in IJC? According to Phil,
>> this option would probably take too long. I'd be interested to hear
>> any other thoughts you might have on publication options.
>
> Hi Ben and Phil,
>
> as you may know (Phil certainly knows), I'm on the editorial board of
> IJC.  Phil is right that it can be rather slow (though faster than
> certain other climate journals!).  Nevertheless, IJC really is the
> preferred place to publish (though a downside is that Douglass et al.
> may have the opportunity to have a response considered to accompany any
> comment).
>
> I just contacted the editor, Glenn McGregor, to see what he can do.  He
> promises to do everything he can to achieve a quick turn-around time (he
> didn't quantify this) and he will also "ask (the publishers) for
> priority in terms of getting the paper online asap after the authors
> have received proofs".  He genuinely seems keen to correct the
> scientific record as quickly as possible.
>
> He also said (and please treat this in confidence, which is why I
> emailed to you and Phil only) that he may be able to hold back the
> hardcopy (i.e. the print/paper version) appearance of Douglass et al.,
> possibly so that any accepted Santer et al. comment could appear
> alongside it.  Presumably depends on speed of the review process.
>
> If this does persuade you to go with IJC, Glenn suggested that I could
> help (because he is in Kathmandu at present) with achieving the quick
> turn-around time by identifying in advance reviewers who are both
> suitable and available.  Obviously one reviewer could be someone who is
> already familiar with this discussion, because that would enable a fast
> review - i.e., someone on the email list you've been using - though I
> don't know which of these people you will be asking to be co-authors and
> hence which won't be available as possible reviewers.  For objectivity
> the other reviewer would need to be independent, but you could still
> suggest suitable names.
>
> Well, that's my thoughts... let me know what you decide.
>
> Cheers
>
> Tim
>
>
> Dr Timothy J Osborn, Academic Fellow
> Climatic Research Unit
> School of Environmental Sciences
> University of East Anglia
> Norwich  NR4 7TJ, UK
>
> e-mail:   t.osborn@uea.ac.uk
> phone:    +44 1603 592089
> fax:      +44 1603 507784
> web:      ~timo/
> sunclock: ~timo/sunclock.htm
>
--
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel:   (925) 422-2486
FAX:   (925) 422-7675
email: santer1@llnl.gov
----------------------------------------------------------------------------


All,
as it happens I am preparing a figure precisely as Dian suggested. This
has only been possible due to substantial efforts by Leo in particular,
but all the other dataset providers also. I wanted to give a feel for
where we are at although I want to tidy this substantially if we were to
use it. To do this I've taken every single scrap of info I have in my
possession that has a status of at least submitted to a journal. I have
considered the common period of 1979-2004. So, assuming you are all
sitting comfortably:
Grey shading is a little cheat from Santer et al using a trusty ruler.
See Figure 3.B in this paper, take the absolute range of model scaling
factors at each of the heights on the y-axis and apply this scaling to
HadCRUT3 tropical mean trend denoted by the star at the surface. So, if
we assume HadCRUT3 is correct then we are aiming for the grey shading or
not depending upon one's pre-conceived notion as to whether the models
are correct.
Red is HadAT2 dataset.
black dashed is the raw data used in Titchner et al. submitted (all
tropical stations with a 81-2000 climatology)
Black whiskers are median, inter-quartile range and max / min from
Titchner et al. submission. We know, from complex error-world
assessments, that the median under-cooks the required adjustment here
and that the truth may conceivably lie (well) outside the upper limit.
Bright green is RATPAC
Then, and the averaging and trend calculation has been done by Leo here
and not me so any final version I'd want to get the raw gridded data and
do it exactly the same way. But for the raw raobs data that Leo provided
as a sanity check it seems to make a miniscule (<0.05K/decade even at
height) difference:
Lime green: RICH (RAOBCORE 1.4 breaks, neighbour based adjustment
estimates)
Solid purple: RAOBCORE 1.2
Dotted purple: RAOBCORE 1.3
Dashed purple: RAOBCORE 1.4
I am also in possession of Steve's submitted IUK dataset and will be
adding this trend line shortly.
I'll be adding a legend in the large white space bottom left.
My take home is that all datasets are heading the right way and that
this reduces the probability of a discrepancy. Compare this with Santer
et al. Figure 3.B.
I'll be using this in an internal report anyway but am quite happy for
it to be used in this context too if that is the general feeling. Or for
Leo's to be used. Whatever people prefer.
Peter
--
Peter Thorne   Climate Research Scientist
Met Office Hadley Centre, FitzRoy Road, Exeter, EX1 3PB
tel. +44 1392 886552 fax +44 1392 885681
www.metoffice.gov.uk/hadobs
Attachment Converted: "c:\eudora\attach\trend_profiles_dogs_dinner.png"


Ben,
Tim's idea is a possibility. I've not always got on that well great
with Glenn McGregor, but Tim seems to have a reasonable rapport
with him. Dian has suggested that this would be the best route - it
is the logical one. I also think that Glenn would get quick reviews, as
Tim thinks he realises he's made a mistake.
Tim has let me into part of secret. Glenn said the paper had two
reviews - one positive, the other said it wasn't great, but would leave it
up to the editor's discretion. This is why Glenn knows he made the wrong
choice.
The problem !! The person who said they would leave it to the editor's
discretion is on your email list!  I don't know who it is - Tim does -
maybe they have told you? I don't want to put pressure on Tim. He
doesn't know I'm sending this. It isn't me by the way - nor Tim !
Tim said it was someone who hasn't contributed to the discussion -
which does narrow the possibilities down!
Tim/Glenn discussed getting quick reviews. Whoever this person
is they could be the familiar reviewer - and we could then come up
with another reasonable name (Kevin - he does everything at the
speed of light) as the two reviewers.
Colour in IJC costs a bit, but I'm sure we can lean on Glenn.
Also we can just have colour in the pdf.
I'll now send a few thoughts on the figures!
Cheers
Phil
Tom Wigley
, Karl Taylor
,
Thomas R Karl
,
John Lanzante
, carl mears
,
"David C. Bader"
,
"'Francis W. Zwiers'"
,
Frank Wentz
,
Leopold Haimberger
,
Melissa Free
,
"Michael C. MacCracken"
,
"'Philip D. Jones'"
,
Steven Sherwood
,
Steve Klein
, 'Susan Solomon'
,
"Thorne, Peter"
,
Tim Osborn
, Gavin Schmidt
,
"Hack, James J."
X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9
Date: Thu, 10 Jan 2008 13:00:39 +0000
To: santer1@llnl.gov,"'Philip D. Jones'"
From: Tim Osborn
Subject: Re: Update on response to Douglass et al.
At 03:52 10/01/2008, Ben Santer wrote:
...Much as I would like to see a high-profile rebuttal of Douglass et al. in a journal
like Science or Nature, it's unlikely that either journal will publish such a rebuttal.
So what are our options? Personally, I'd vote for GRL. I think that it is important to
publish an expeditious response to the statistical flaws in Douglass et al. In theory,
GRL should be able to give us the desired fast turnaround time...
Why not go for publication of a response in IJC? According to Phil, this option would
probably take too long. I'd be interested to hear any other thoughts you might have on
publication options.
Hi Ben and Phil,
as you may know (Phil certainly knows), I'm on the editorial board of IJC.  Phil is
right that it can be rather slow (though faster than certain other climate journals!).
Nevertheless, IJC really is the preferred place to publish (though a downside is that
Douglass et al. may have the opportunity to have a response considered to accompany any
comment).
I just contacted the editor, Glenn McGregor, to see what he can do.  He promises to do
everything he can to achieve a quick turn-around time (he didn't quantify this) and he
will also "ask (the publishers) for priority in terms of getting the paper online asap
after the authors have received proofs".  He genuinely seems keen to correct the
scientific record as quickly as possible.
He also said (and please treat this in confidence, which is why I emailed to you and
Phil only) that he may be able to hold back the hardcopy (i.e. the print/paper version)
appearance of Douglass et al., possibly so that any accepted Santer et al. comment could
appear alongside it.  Presumably depends on speed of the review process.
If this does persuade you to go with IJC, Glenn suggested that I could help (because he
is in Kathmandu at present) with achieving the quick turn-around time by identifying in
advance reviewers who are both suitable and available.  Obviously one reviewer could be
someone who is already familiar with this discussion, because that would enable a fast
review - i.e., someone on the email list you've been using - though I don't know which
of these people you will be asking to be co-authors and hence which won't be available
as possible reviewers.  For objectivity the other reviewer would need to be independent,
but you could still suggest suitable names.
Well, that's my thoughts... let me know what you decide.
Cheers
Tim
Dr Timothy J Osborn, Academic Fellow
Climatic Research Unit
School of Environmental Sciences
University of East Anglia
Norwich  NR4 7TJ, UK
e-mail:   t.osborn@uea.ac.uk
phone:    +44 1603 592089
fax:      +44 1603 507784
web:      [1]~timo/
sunclock: [2]~timo/sunclock.htm
Prof. Phil Jones
Climatic Research Unit        Telephone +44 (0) 1603 592090
School of Environmental Sciences    Fax +44 (0) 1603 507784
University of East Anglia
Norwich                          Email    p.jones@uea.ac.uk
NR4 7TJ
UK
----------------------------------------------------------------------------
References
1. ~timo/
2. ~timo/sunclock.htm


Ben et al,
As Dian has said Ben's diagrams are as usual great! I also like the one
that Peter has just sent around as that illustrates the issue with the
various RAOBCORE versions. Although I still think they should have used
HadCRUT3v for the surface, I know HadCRUT2v shows much the same.
What this figure shows is the differences between the various sonde
datasets. Dian/Peter also make the point that there are other new datasets
to be added - so the sondes are very much still work in progress. I know
you will point out all the analytical/statistical issues see the series
brings home the issues better. I know you could add the values to
your Fig1, a plot like this is much better.
In the email Ben, you seem to have written much of the response!
Whichever route you go down (GRL/IJC) the text can't be too long.
I would favour copious captions, and even an Appendix, to get the
main points across quickly.
Cheers
Phil
At 14:43 10/01/2008, Peter Thorne wrote:
All,
as it happens I am preparing a figure precisely as Dian suggested. This
has only been possible due to substantial efforts by Leo in particular,
but all the other dataset providers also. I wanted to give a feel for
where we are at although I want to tidy this substantially if we were to
use it. To do this I've taken every single scrap of info I have in my
possession that has a status of at least submitted to a journal. I have
considered the common period of 1979-2004. So, assuming you are all
sitting comfortably:
Grey shading is a little cheat from Santer et al using a trusty ruler.
See Figure 3.B in this paper, take the absolute range of model scaling
factors at each of the heights on the y-axis and apply this scaling to
HadCRUT3 tropical mean trend denoted by the star at the surface. So, if
we assume HadCRUT3 is correct then we are aiming for the grey shading or
not depending upon one's pre-conceived notion as to whether the models
are correct.
Red is HadAT2 dataset.
black dashed is the raw data used in Titchner et al. submitted (all
tropical stations with a 81-2000 climatology)
Black whiskers are median, inter-quartile range and max / min from
Titchner et al. submission. We know, from complex error-world
assessments, that the median under-cooks the required adjustment here
and that the truth may conceivably lie (well) outside the upper limit.
Bright green is RATPAC
Then, and the averaging and trend calculation has been done by Leo here
and not me so any final version I'd want to get the raw gridded data and
do it exactly the same way. But for the raw raobs data that Leo provided
as a sanity check it seems to make a miniscule (<0.05K/decade even at
height) difference:
Lime green: RICH (RAOBCORE 1.4 breaks, neighbour based adjustment
estimates)
Solid purple: RAOBCORE 1.2
Dotted purple: RAOBCORE 1.3
Dashed purple: RAOBCORE 1.4
I am also in possession of Steve's submitted IUK dataset and will be
adding this trend line shortly.
I'll be adding a legend in the large white space bottom left.
My take home is that all datasets are heading the right way and that
this reduces the probability of a discrepancy. Compare this with Santer
et al. Figure 3.B.
I'll be using this in an internal report anyway but am quite happy for
it to be used in this context too if that is the general feeling. Or for
Leo's to be used. Whatever people prefer.
Peter
--
Peter Thorne   Climate Research Scientist
Met Office Hadley Centre, FitzRoy Road, Exeter, EX1 3PB
tel. +44 1392 886552 fax +44 1392 885681
[1]www.metoffice.gov.uk/hadobs
Prof. Phil Jones
Climatic Research Unit        Telephone +44 (0) 1603 592090
School of Environmental Sciences    Fax +44 (0) 1603 507784
University of East Anglia
Norwich                          Email    p.jones@uea.ac.uk
NR4 7TJ
UK
----------------------------------------------------------------------------
References
1.


Dear Leo,
Thanks very much for your email. I can easily make the observations a
bit more prominent in Figure 1. As you can see from today's
(voluminous!) email traffic, I've received lots of helpful suggestions
regarding improvements to the Figures. I'll try to produce revised
versions of the Figures tomorrow.
On the autocorrelation issue: The models have a much larger range of
lag-1 autocorrelation coefficients (0.66 to 0.95 for T2LT, and 0.69 to
0.95 for T2) than the UAH or RSS data (which range from 0.87 to 0.89). I
was concerned that if we used the model lag-1 autocorrelations to guide
the choice of AR-1 parameter in the synthetic data analysis, Douglass
and colleagues would have an easy opening for criticising us ("Aha!
Santer et al. are using model results to guide them in their selection
of the coefficients for their AR-1 model!") I felt that it was much more
difficult for Douglass et al. to criticize what we've done if we used
UAH data to dictate our choice of the AR-1 parameter and the "scaling
factor" for the amplitude of the temporal variability.
As you know, my personal preference would be to include in our response
to Douglass et al. something like the Figure 4 that Peter has produced.
While inclusion of a Figure 4 is not essential for the purpose of
illuminating the statistical flaws in the Douglass et al. "consistency
test", such a Figure would clearly show the (currently large) structural
uncertainties in radiosonde-based estimates of the vertical profile of
atmospheric temperature changes. I think this is an important point,
particularly in view of the fact that Douglass et al. failed to discuss
versions 1.3 and 1.4 of your RAOBCORE data - even though they had
information from those datasets in their possession.
However, I fully agree with Tom's comment that we don't want to do
anything to "steal the thunder" from ongoing efforts to improve
sonde-based estimates of atmospheric temperature change, and to better
quantify structural uncertainties in those estimates. Your group,
together with the groups at the Hadley Centre, Yale, NOAA ARL and NOAA
GFDL, deserve great credit for making significant progress on a
difficult, time-consuming, yet important problem.
I guess the best solution is to leave this decision up to all of you
(the radiosonde dataset developers). I'm perfectly happy to include a
version of Figure 4 in our response to Douglass et al. If we do go with
inclusion of a Figure 4, you, Peter, Dian, Melissa, Steve Sherwood and
John should decide whether you feel comfortable providing radiosonde
data for such a Figure. I will gladly abide by your decisions. As you
note in your email, our use of a Figure 4 would not preclude a more
detailed and thorough comparison of simulated and observed amplification
in some later publication.
Once again, thanks for all your help with this project, Leo.
With best regards,
Ben
Leopold Haimberger wrote:
> All,
>
> These three figures are really very clear and leave no doubts that the
> Douglass et al analysis is flawed. This is true especially for Fig. 1.
> In Fig. 1 one has to look carefully to find the RSS and UAH "observed"
> trends to the right of all the model trends. Maybe one can make their
> symbols more prominent.
>
> Concerning Fig. 3 I wonder whether the UAH autocorrelation is the lowest
> of all available data. .86 is quite substantial autocorrelation. Maybe
> it is a good idea to be on the safe side and use the lowest
> autocorrelation of all datasets (models, RSS, UAH) for this analysis.
>
> Concerning Fig. 4, I like Peter's and Dian's idea to include RAOBCORE,
> HadAT2, RATPAC and Steve's data and compare it in one plot with model
> output. While I agree that the first three figures and the corresponding
> text are already sufficient for the reply, they target mainly to the
> right panel of Fig. 1 in Douglass et al's paper. The trend profile plot
> of Fig. 4 is complementary  as a counterpart to the left panel of their
> plot. To see the trend amplification in in some of the vertical profiles
> is much more suggestive than seeing the LT trends being larger than
> surface trends, at least for me. Showing all available profiles adds
> value beyond the RAOBCORE v1.2 vs RAOBCORE v1.4 issue. Yes, it is work
> in progress and such a plot as drafted by Peter makes that very clear.
> In this paper it is sufficient to show that the uncertainty of
> radiosonde trends is much larger than suggested by Douglass et al. and
> we do not need to have the final answer yet. I have nothing against
> Peter doing the drawing of the figure, since he has most of the
> necessary data. The plot would be needed for 1979-1999, however. Peter,
> I will send you the trend profiles for this period a bit later.
>
> Publishing the reply in either IJC or GRL including Fig. 4 is fine for me.
> When we first discussed a follow up of the Santer et al paper in
> October, we had in mind to publish post-FAR climate model data up to
> present (not just 1999) and also new radiosonde data up to present in a
> highest ranking journal. I am confident that this is still possible even
> if some of the new material planned for such a paper is submitted
> already now. What do you think?
>
> With best Regards,
>
> Leo
>
> Peter Thorne wrote:
>> All,
>>
>> as it happens I am preparing a figure precisely as Dian suggested. This
>> has only been possible due to substantial efforts by Leo in particular,
>> but all the other dataset providers also. I wanted to give a feel for
>> where we are at although I want to tidy this substantially if we were to
>> use it. To do this I've taken every single scrap of info I have in my
>> possession that has a status of at least submitted to a journal. I have
>> considered the common period of 1979-2004. So, assuming you are all
>> sitting comfortably:
>>
>> Grey shading is a little cheat from Santer et al using a trusty ruler.
>> See Figure 3.B in this paper, take the absolute range of model scaling
>> factors at each of the heights on the y-axis and apply this scaling to
>> HadCRUT3 tropical mean trend denoted by the star at the surface. So, if
>> we assume HadCRUT3 is correct then we are aiming for the grey shading or
>> not depending upon one's pre-conceived notion as to whether the models
>> are correct.
>>
>> Red is HadAT2 dataset.
>>
>> black dashed is the raw data used in Titchner et al. submitted (all
>> tropical stations with a 81-2000 climatology)
>>
>> Black whiskers are median, inter-quartile range and max / min from
>> Titchner et al. submission. We know, from complex error-world
>> assessments, that the median under-cooks the required adjustment here
>> and that the truth may conceivably lie (well) outside the upper limit.
>>
>> Bright green is RATPAC
>>
>> Then, and the averaging and trend calculation has been done by Leo here
>> and not me so any final version I'd want to get the raw gridded data and
>> do it exactly the same way. But for the raw raobs data that Leo provided
>> as a sanity check it seems to make a miniscule (
> height) difference:
>>
>> Lime green: RICH (RAOBCORE 1.4 breaks, neighbour based adjustment
>> estimates)
>>
>> Solid purple: RAOBCORE 1.2
>> Dotted purple: RAOBCORE 1.3
>> Dashed purple: RAOBCORE 1.4
>>
>> I am also in possession of Steve's submitted IUK dataset and will be
>> adding this trend line shortly.
>>
>> I'll be adding a legend in the large white space bottom left.
>>
>> My take home is that all datasets are heading the right way and that
>> this reduces the probability of a discrepancy. Compare this with Santer
>> et al. Figure 3.B.
>>
>> I'll be using this in an internal report anyway but am quite happy for
>> it to be used in this context too if that is the general feeling. Or for
>> Leo's to be used. Whatever people prefer.
>>
>> Peter
>>
>>
>> ------------------------------------------------------------------------
>>
>
--
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel:   (925) 422-2486
FAX:   (925) 422-7675
email: santer1@llnl.gov
----------------------------------------------------------------------------


Dear Tim,
Here are some suggestions for potential reviewers of a Santer et al.
IJoC submission on issues related to the consistency between modeled and
observed atmospheric temperature trends. None of the suggested reviewers
have been involved in the recent "focus group" that has discussed
problems with the Douglass et al. IJoC paper.
1. Mike Wallace, University of Washington. U.S. National Academy member.
Expert on atmospheric dynamics. Chair of National Academy of Sciences
committee on "Reconciling observations of global temperature change"
(2000). Email: wallace@atmos.washington.edu
2. Qiang Fu, University of Washington. Expert on atmospheric radiation,
dynamics, radiosonde and satellite data. Published 2004 Nature paper and
2005 GRL paper dealing with issues related to global and tropical
temperature trends. Email: qfu@atmos.washington.edu
3. Gabi Hegerl, University of Edinburgh. Expert on detection and
attribution of externally-forced climate change. Co-Convening Lead
Author of "Understanding and Attributing Climate Change" chapter of IPCC
Fourth Assessment Report. Email: Gabi.Hegerl@ed.ac.uk
4. Jim Hurrell, National Center for Atmospheric Research (NCAR). Former
Director of Climate and Global Dynamics division at NCAR. Expert on
climate modeling, observational data. Published a number of papers on
MSU-related issues. Email: jhurrell@cgd.ucar.edu
5. Myles Allen, Oxford University. Expert in Climate Dynamics, detection
and attribution, application of statistical methods in climatology.
Email: allen@atm.ox.ac.uk
6. Peter Stott, Hadley Centre for Climate Prediction and Research.
Expert in climate modeling, detection and attribution. Email:
peter.stott@metoffice.gov.uk
With best regards,
Ben
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel:   (925) 422-2486
FAX:   (925) 422-7675
email: santer1@llnl.gov
----------------------------------------------------------------------------


Hi Ben (cc Phil),
just heard back from Glenn.  He's prepared to treat it as a new
submission rather than a comment on Douglass et al. and he also
reiterates that "Needless to say my offer of a quick turn around time
etc still stands".
So basically this makes the IJC option more attractive than if it
were treated as a comment.  But whether IJC is still a less
attractive option than GRL is up to you to decide :-) (or feel free
to canvas your potential co-authors [the only thing I didn't want to
make more generally known was the suggestion that print publication
of Douglass et al. might be delayed... all other aspects of this
discussion are unrestricted]).
Cheers
Tim
At 21:00 10/01/2008, Ben Santer wrote:
>Dear Tim,
>
>Thanks very much for your email. I greatly appreciate the additional
>information that you've given me. I am a bit conflicted about what
>we should do.
>
>IJC published a paper with egregious statistical errors. Douglass et
>al. was essentially a commentary on work by myself and colleagues -
>work that had been previously published in Science in 2005 and in
>Chapter 5 of the first U.S. CCSP Report in 2006. To my knowledge,
>none of the authors or co-authors of the Santer et al. Science paper
>or of CCSP 1.1 Chapter 5 were used as reviewers of Douglass et al. I
>am assuming that, when he submitted his paper to IJC, Douglass
>specifically requested that certain scientists should be excluded
>from the review process. Such an approach is not defensible for a
>paper which is largely a comment on previously-published work.
>
>It would be fair and reasonable to give IJC the opportunity to "set
>the record straight", and correct the harm they have done by
>publication of Douglass et al. I use the word "harm" advisedly. The
>author and coauthors of the Douglass et al. IJC paper are using this
>paper to argue that "Nature, not CO2, rules the climate", and that
>the findings of Douglass et al. invalidate the "discernible human
>influence" conclusions of previous national and international
>scientific assessments.
>
>Quick publication of a response to Douglass et al. in IJC would go
>some way towards setting the record straight. I am troubled,
>however, by the very real possibility that Douglass et al. will have
>the last word on this subject. In my opinion (based on many years of
>interaction with these guys), neither Douglass, Christy or Singer
>are capable of admitting that their paper contained serious
>scientific errors. Their "last word" will be an attempt to obfuscate
>rather than illuminate. They are not interested in improving our
>scientific understanding of the nature and causes of recent changes
>in atmospheric temperature. They are solely interested in advancing
>their own agendas. It is telling and troubling that Douglass et al.
>ignored radiosonde data showing substantial warming of the tropical
>troposphere - data that were in accord with model results - even
>though such data were in their possession. Such behaviour
>constitutes intellectual dishonesty. I strongly believe that leaving
>these guys the last word is inherently unfair.
>
>If IJC are interested in publishing our contribution, I believe it's
>fair to ask for the following:
>
>1) Our paper should be regarded as an independent contribution, not
>as a comment on Douglass et al. This seems reasonable given i) The
>substantial amount of new work that we have done; and ii) The fact
>that the Douglass et al. paper was not regarded as a comment on
>Santer et al. (2005), or on Chapter 5 of the 2006 CCSP Report - even
>though Douglass et al. clearly WAS a comment on these two publications.
>
>2) If IJC agrees to 1), then Douglass et al. should have the
>opportunity to respond to our contribution, and we should be given
>the chance to reply. Any response and reply should be published
>side-by-side, in the same issue of IJC.
>
>I'd be grateful if you and Phil could provide me with some guidance
>on 1) and 2), and on whether you think we should submit to IJC. Feel
>free to forward my email to Glenn McGregor.
>
>With best regards,
>
>Ben
>Tim Osborn wrote:
>>At 03:52 10/01/2008, Ben Santer wrote:
>>>...Much as I would like to see a high-profile rebuttal of Douglass
>>>et al. in a journal like Science or Nature, it's unlikely that
>>>either journal will publish such a rebuttal.
>>>
>>>So what are our options? Personally, I'd vote for GRL. I think
>>>that it is important to publish an expeditious response to the
>>>statistical flaws in Douglass et al. In theory, GRL should be able
>>>to give us the desired fast turnaround time...
>>>
>>>Why not go for publication of a response in IJC? According to
>>>Phil, this option would probably take too long. I'd be interested
>>>to hear any other thoughts you might have on publication options.
>>Hi Ben and Phil,
>>as you may know (Phil certainly knows), I'm on the editorial board
>>of IJC.  Phil is right that it can be rather slow (though faster
>>than certain other climate journals!).  Nevertheless, IJC really is
>>the preferred place to publish (though a downside is that Douglass
>>et al. may have the opportunity to have a response considered to
>>accompany any comment).
>>I just contacted the editor, Glenn McGregor, to see what he can
>>do.  He promises to do everything he can to achieve a quick
>>turn-around time (he didn't quantify this) and he will also "ask
>>(the publishers) for priority in terms of getting the paper online
>>asap after the authors have received proofs".  He genuinely seems
>>keen to correct the scientific record as quickly as possible.
>>He also said (and please treat this in confidence, which is why I
>>emailed to you and Phil only) that he may be able to hold back the
>>hardcopy (i.e. the print/paper version) appearance of Douglass et
>>al., possibly so that any accepted Santer et al. comment could
>>appear alongside it.  Presumably depends on speed of the review process.
>>If this does persuade you to go with IJC, Glenn suggested that I
>>could help (because he is in Kathmandu at present) with achieving
>>the quick turn-around time by identifying in advance reviewers who
>>are both suitable and available.  Obviously one reviewer could be
>>someone who is already familiar with this discussion, because that
>>would enable a fast review - i.e., someone on the email list you've
>>been using - though I don't know which of these people you will be
>>asking to be co-authors and hence which won't be available as
>>possible reviewers.  For objectivity the other reviewer would need
>>to be independent, but you could still suggest suitable names.
>>Well, that's my thoughts... let me know what you decide.
>>Cheers
>>Tim
>>
>>Dr Timothy J Osborn, Academic Fellow
>>Climatic Research Unit
>>School of Environmental Sciences
>>University of East Anglia
>>Norwich  NR4 7TJ, UK
>>e-mail:   t.osborn@uea.ac.uk
>>phone:    +44 1603 592089
>>fax:      +44 1603 507784
>>web:      ~timo/
>>sunclock: ~timo/sunclock.htm
>
>
>--
>----------------------------------------------------------------------------
>Benjamin D. Santer
>Program for Climate Model Diagnosis and Intercomparison
>Lawrence Livermore National Laboratory
>P.O. Box 808, Mail Stop L-103
>Livermore, CA 94550, U.S.A.
>Tel:   (925) 422-2486
>FAX:   (925) 422-7675
>email: santer1@llnl.gov
>----------------------------------------------------------------------------
Dr Timothy J Osborn, Academic Fellow
Climatic Research Unit
School of Environmental Sciences
University of East Anglia
Norwich  NR4 7TJ, UK
e-mail:   t.osborn@uea.ac.uk
phone:    +44 1603 592089
fax:      +44 1603 507784
web:      ~timo/
sunclock: ~timo/sunclock.htm


I didn't know about the link between John and Kevin.  Sounds like Qiang or Myles, plus
Francis, would be best combination of expertise and speediness.
By the way, for online submission you'll just need to convert the Latex to a PDF file and
submit that.
Have a good weekend,
Tim
At 17:07 11/01/2008, you wrote:
Dear Phil and Tim,
I did leave Kevin's name off because of concerns that he might be extremely upset by
Christy's involvement in Douglass et al. I guess you know that John was a Ph.D. student
of Kevin's. It must be tough to have a student who's the antithesis of everything you
stand for and care about - careful, thorough science.
Qiang Fu would be great, since he's so knowledgable about MSU-related issues. I think he
would be fast, too. Myles reviewed one of the GRL versions of Douglass et al., so he's
very familiar with this territory.
With best regards,
Ben
Phil Jones wrote:
Ben,
I briefly discussed this with Tim a few minutes ago.
With IDAG coming up, it is probably best not on to use Gabi and Myles.
I also suggested that Mike Wallace might be slow - as Myles would
have been. Peter S might not be right for the IDAG reason and he
does work for the HC - where Peter T does.
If Jim is back working he would be good. So would Fu. If Tim can
just persuade them to do it - and quickly.
I did suggest Kevin - he would do it quickly - but it may be a read rag
to a bull with John Christy on the other paper.
Glad to see you've gone down his route!
Have a good weekend!
Ruth says hello!
Cheers
Phil
At 16:43 11/01/2008, Ben Santer wrote:
Dear Tim,
Here are some suggestions for potential reviewers of a Santer et al. IJoC submission on
issues related to the consistency between modeled and observed atmospheric temperature
trends. None of the suggested reviewers  have been involved in the recent "focus group"
that has discussed problems with the Douglass et al. IJoC paper.
1. Mike Wallace, University of Washington. U.S. National Academy member. Expert on
atmospheric dynamics. Chair of National Academy of Sciences committee on "Reconciling
observations of global temperature change" (2000). Email: wallace@atmos.washington.edu
2. Qiang Fu, University of Washington. Expert on atmospheric radiation, dynamics,
radiosonde and satellite data. Published 2004 Nature paper and 2005 GRL paper dealing
with issues related to global and tropical temperature trends. Email:
qfu@atmos.washington.edu
3. Gabi Hegerl, University of Edinburgh. Expert on detection and attribution of
externally-forced climate change. Co-Convening Lead Author of "Understanding and
Attributing Climate Change" chapter of IPCC Fourth Assessment Report. Email:
Gabi.Hegerl@ed.ac.uk
4. Jim Hurrell, National Center for Atmospheric Research (NCAR). Former Director of
Climate and Global Dynamics division at NCAR. Expert on climate modeling, observational
data. Published a number of papers on MSU-related issues. Email: jhurrell@cgd.ucar.edu
5. Myles Allen, Oxford University. Expert in Climate Dynamics, detection and
attribution, application of statistical methods in climatology. Email:
allen@atm.ox.ac.uk
6. Peter Stott, Hadley Centre for Climate Prediction and Research. Expert in climate
modeling, detection and attribution. Email: peter.stott@metoffice.gov.uk
With best regards,
Ben
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel:   (925) 422-2486
FAX:   (925) 422-7675
email: santer1@llnl.gov
----------------------------------------------------------------------------
Prof. Phil Jones
Climatic Research Unit        Telephone +44 (0) 1603 592090
School of Environmental Sciences    Fax +44 (0) 1603 507784
University of East Anglia
Norwich                          Email    p.jones@uea.ac.uk
NR4 7TJ
UK
----------------------------------------------------------------------------
--
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel:   (925) 422-2486
FAX:   (925) 422-7675
email: santer1@llnl.gov
----------------------------------------------------------------------------


Dear folks,
I believe Ben's suggestion is very  good compromise and we should
prepare a Fig. 4 with three RAOBCORE versions, RICH, HadAT and RATPAC.
As I have understood Ben in his first description of Fig. 4, also the
range of model trend profiles should be included.
Who will actually draw the figure? I can do this but I do not have the
model data and I do not have the RATPAC profiles so far. It would be
easiest to remove the Titchner et al. profiles and Steves profiles from
Peter's plot. Or should we send our profile data to you, Ben? What do
you think?
Concerning the possible reaction of Douglass et al.: RAOBCORE v1.2 and
v1.3 are both published in the Haimberger(2007) RAOBCORE paper (where
they were labeled differently). Thus they have at least omitted v1.3.
RAOBCORE v1.4 time series have published in the May 2007 BAMS climate
state of 2006 supplement.
Peter, myself, Dian and probably a few others will meet in Japan by the
End of January and a few weeks later in Germany, where we can discuss
the latest developments  and plan the publishing strategy.
Thanks a lot Ben for moderating this Fig. 4 issue.
Regards,
Leo
Ben Santer wrote:
> Dear folks,
>
> Just a quick update. With the assistance of Tim Osborn, Phil Jones, and
> Dian, I've now come to a decision about the disposition of our response
> to Douglass et al. I've decided to submit to IJoC. I think this is a
> fair and reasonable course of action. The IJoC editor (and various IJoC
> editorial board members and Royal Meteorological Society members) now
> recognize that the Douglass et al. paper contains serious statistical
> flaws, and that its publication in IJoC reflects poorly on the IJoC and
> Royal Meteorological Society. From my perspective, IJoC should be given
> the opportunity to set the record straight.
>
> The editor of IJoC, Glenn McGregor, has agreed to treat our paper as an
> independent submission rather than as a comment on Douglass et al. This
> avoids the situation that I was afraid of - that our paper would be
> viewed as a comment, and Douglass et al. would have the "last word" in
> this exchange. In my opinion (based on many years of interaction with
> these guys), neither Douglass, Christy or Singer are capable of
> admitting that their paper contained serious scientific errors. Their
> "last word" would have been an attempt to obfuscate rather than
> illuminate. That would have been very unfortunate.
>
> If our contribution is published in IJoC, Douglass et al. will have the
> opportunity to comment on it, and we will have the right to reply.
> Ideally, any comment and reply should be published side-by-side in the
> same issue of IJoC.
>
> The other good news is that IJoC is prepared to handle our submission
> expeditiously. My target, therefore, is to finalize our submission by
> the end of next week. I hope to have a first draft to send you by no
> later than next Tuesday.
>
> Now on to the "Figure 4" issue. Thanks to many of you for very helpful
> discussions and advice. Here are some comments:
>
> 1) I think it is important to have a Figure 4. We need to provide
> information on structural uncertainties in radiosonde-based estimates of
> profiles of atmospheric temperature change. Douglass et al. did not
> accurately portray the full range of structural uncertainties.
>
> 2) I do not want our submission to detract from other publications
> dealing with recent progress in the development of sonde-based
> atmospheric temperature datasets. I am aware of at least four such
> publications which are "in the pipeline".
>
> 3) So here is my suggestion for a compromise.
>
> o   If Leo is agreeable, I would like to show results from his three
> RAOBCORE versions (v1.2, v1.3, and v1.4) in Figure 4. I'd also like to
> include results from the RATPAC and HadAT datasets used by Douglass et
> al. This allows us to illustrate that Douglass et al. were highly
> selective in their choice of radiosonde data. They had access to results
> from all three versions of RAOBCORE, but chose to show results from v1.2
> only - the version that provided the best support for their "models are
> inconsistent with observations" argument.
>
> o   I suggest that we do NOT show the most recent radiosonde results
> from the Hadley Centre (described in the Titchner et al. paper) or from
> Steve Sherwood's group. This leaves more scope for a subsequent paper
> along the lines suggested by Leo, which would synthesize the results
> from the very latest sonde- and satellite-based temperature datasets,
> and compare these results with model-based estimates of atmospheric
> temperature change. I think that someone from the sonde community should
> take the lead on such a paper.
>
> 4) As Melissa has pointed out, Douglass et al. may argue that v1.2 was
> published at the time they wrote their paper, while v1.3 and v1.4 were
> unpublished (but submitted). I'm sure this is how Douglass et al. will
> actually respond. Nevertheless, I strongly believe that Douglass et al.
> should have at least mentioned the existence of the v1.3 and v1.4 results.
>
> Do these suggested courses of action (submission to IJoC and inclusion
> of a Figure 4 with RAOBCOREv1.2,v1.3,v1.4/RATPAC/HadAT data) sound
> reasonable to you?
>
> With best regards,
>
> Ben
> ----------------------------------------------------------------------------
>
> Benjamin D. Santer
> Program for Climate Model Diagnosis and Intercomparison
> Lawrence Livermore National Laboratory
> P.O. Box 808, Mail Stop L-103
> Livermore, CA 94550, U.S.A.
> Tel:   (925) 422-2486
> FAX:   (925) 422-7675
> email: santer1@llnl.gov
> ----------------------------------------------------------------------------
>
--
Ao. Univ. Prof. Dr. Leopold Haimberger
Institut für Meteorologie und Geophysik, Universität Wien
Althanstraße 14, A - 1090 Wien
Tel.: +43 1 4277 53712
Fax.: +43 1 4277 9537
~haimbel7/