Model Noise Revisited – cont’d

This post will deal with a different method of calculating weather noise than was used in this previous post. Responding to comments in previous posts and reviewing what I’ve written in the past has made me realize that I have not done a decent job communicating exactly what I’m doing. Therefore, starting with this post, I will sketch out step-by-step exactly what I’m doing so others can replicate it to check the veracity of my conclusions.

The previous method used to calculate weather noise in the models and surface temperature record (STR) involved fitting a trend line through the data and using the residuals to compute the standard error. The standard error is the ratio of the standard deviation of the residuals divided by the square root of the number of observations. The standard deviation of the residuals is a measure of how far, on average, the residuals differ from the mean of the residuals. Now if the models are accurately reproducing the chaotic, seemingly random fluctuations we see in the atmosphere/ocean system over short periods of time, then their standard errors and the spread of such errors should be in line with what is observed in the surface temperature record. My method differs from Lucia’s in that instead of fitting a trend line through the data, I smoothed it on a time span that would remove a great deal of the random peaks and valleys in the data. By subtracting this from the original time series, we get a more realistic depiction of the noise in the signal that can be attributed to the short term variation (weather).

Calculations

1. First I load the GISS, HadCrut3, and NCDC temperature anomaly data sets. I want all three data sets having a common base period. So I take the monthly mean surface temperatures for the base periods of HadCrut3 and NCDC and use them to transform the temperature anomalies back into temperatures. I then take the data on the range 1951-1980 and calculate the monthly mean surface temperature so I can offset the temperatures back into temperature anomalies. I then take the subset on 1900-2000 for the data I’ll work with.

2. Next I passed the three rebaselined data sets to a function I wrote that will smooth them out on a specified span of years. I used smooth with the option ‘roless’. This option I specified gives zero weight to data that is outside six mean absolute deviations so the wide jumps in temperature anomaly will have little effect on the smoothed function. My function takes these smoothed values and subtracts them from the original data set. It uses the residuals to compute the standard deviation and from that the standard error. I then place all three values of the standard error into a vector.

3. Now we move onto the model data. Thanks to Lucia, I found Climate Explorer which made extracting data from NetCDF files irrelevant. I downloaded the “temperature at surface (tas)” data for EchoG, GISS ER, GISS EH, CGCM 3.1, and CGCM 2.3.2a as well as others, but I’ll discuss these five for this post. These models run scenario 20c3m whose purpose is to simulate the climate of the 20th century. Echo G, CGCM 2.3.2a/3.1 and GISS EH have five ensemble members while GISS ER has nine. The data represents monthly values for global mean surface temperature (GMST) so first we baseline the data using the same base period as our STR. Next we feed the datasets into the same program described in step 2. I then take the subset 1900-2000 for all five data sets.

4. Next I check for autocorrelation (AC) by creating an autocorrelation plot. If it appears that AC is present, I use a partial autocorrelation plot to identify what order. We’ll deal with this issue later.

5. Now it’s time for the hypothesis tests. The hypothesis test I used is defined as

Ho: Standard Error(Earth) = Standard Error(Model)

Ha: Standard Error(Earth) ~= Standard Error(Model)

Standard Error(Earth) is the vector containing the three values for standard error calculated from the STR. Standard Error(Model) is the vector containing all the values calculated from each ensemble member of a particular model. I used ttest2 with the options ‘both’ and ‘unequal’. The option ‘both’ specifies we’re doing a two-tailed test and ‘unequal’ tells the function to not assume both samples have equal variance. I’ve set the significance level to 5%. If autocorrelation is judged to be a problem, then I’ll have to make the appropriate modifications of ttest2.

What about Volcanoes?

A recent comment by Lucia reminded me that it would be a good idea to look into periods of low volcanic activity. I will still look over the range 1900-2000 and to see how well the weather noise tests perform with and without volcanic effects.

Some Results and Graphs

Figure 1 shows the noise calculated from STR and the five models are based on a smoothing on a five year span. Smoothing on 10-,15-, and 20-year spans doesn’t make a real noticeable difference so I won’t bother posting it.

Figure 1

It appears to the eye that the range of variation is significantly larger than what we see in the five models. But let’s look at some numbers to see if our suspicion is correct. The table below summarizes the standard errors in °C for the three STR sources and model ensembles based on 1,5,10,15,20-year smoothing. Since copy and pasting from Excel into WordPress mangles the table in some browsers, I’ve posted screen shots instead.

Echo G and GISS ER consistently fail. The CGCM family passes each time with huge p-values. However, GISS ER would pass each time if we used a 1% significance level! The next post will compare the results from my method to Lucia’s.

Other Methods to Measure Weather Noise

Right now I’m reading a few papers to get a handle on how climate modelers detect weather noise:

Zwiers and Kharin , “Intercomparison of interannual variability and potential predictability: an AMIP diagnostic subproject” Climate Dynamics 1998 14:517-528.

Schneider and Fan, “Weather Noise Forcing of Surface Climate Variability” Journal of the Atmospheric Sciences; 2007, Vol. 64 Issue 9, p3265-328

Roulston and Neelin, “The Response of an ENSO Model to Climate Noise, Weather Noise and Intraseasonal Forcing” GEOPHYSICAL RESEARCH LETTERS, 2000 VOL. 27, NO. 22, PAGES 3723-3726