Model biases can be largely removed using empirical techniques a posteriori!

The title sounds pretty impressive – doesn´t it?

Well – let us translate it into plain english and see what the title really says:

  • Model errors can be removed by adjusting the model so that the output match what is observed

Still impressed?

I´m not impressed. If a supplier try to sell me a model for prediction of a certain variable – and tells me that the output departs from being realistic after a few years, but the model still works fine because it can be adjusted to match the observation – I wouldn´t buy it. However, IPCC seems to be impressed by such model performance. The quote in the title has been cut directly from their report:

«When initialized with states close to the observations, models ‘drift’ towards their imperfect climatology (an estimate of the mean climate), leading to biases in the simulations that depend on the forecast time. The time scale of the drift in the atmosphere and upper ocean is, in most cases, a few years. Biases can be largely removed using empirical techniques a posteriori. …»
(Ref: Contribution from Working Group I to the fifth assessment report by IPCC; 11.2.3 Prediction Quality; 11.2.3.1 Decadal Prediction Experiments )

As phrased by ghl at judithcurry.com:

  • “Biases can be largely removed using empirical techniques a posteriori”
    That’s how I panel beat.
    Seriously, really?

I have been told that the models are reliable. I have been told that the models are based on pure physics. And then I find, as suspected – I must say, that the reason why the model output seems to match the observations is that the models have been adjusted to match the observations.

Here is another quote from the Preface to the report by Working Group 1 to the fifth Assessment Report by IPCC. A quote which implies how important the models has been for the evaluations by IPCC:

The scientific community and the climate modelling centres around the world brought together their activities in the Coordinated Modelling Intercomparison Project Phase 5 (CMIP5), providing the basis for most of the assessment of future climate change in this report. Their efforts enable Working Group I to deliver comprehensive scientific information for the policymakers and the users of this report, as well as for the specific assessments of impacts carried out by IPCC Working Group II, and of costs and mitigation strategies, carried out by IPCC Working Group III.

Here is a slightly edited section from the IPCC report, where I have removed: All references, a few sentences and a few sections to increase readability. I think the section gives some insight into how the models are adjusted to keep the model output in line.

When initialized with states close to the observations, models ‘drift’ towards their imperfect climatology (an estimate of the mean climate), leading to biases in the simulations that depend on the forecast time. The time scale of the drift in the atmosphere and upper ocean is, in most cases, a few years. Biases can be largely removed using empirical techniques a posteriori. The bias correction or adjustment linearly corrects for model drift. The approach assumes that the model bias is stable over the prediction period (from 1960 onward in the CMIP5 experiment). This might not be the case if, for instance, the predicted temperature trend differs from the observed trend.

Figure 11.2 is an illustration of the time scale of the global SST drift, while at the same time showing the systematic error of several of the forecast systems contributing to CMIP5. It is important to note that the systematic errors illustrated here are common to both decadal prediction systems and climate-change projections.

To reduce the impact of the drift many of the early attempts at decadal prediction use an approach called anomaly initialization. The anomaly initialization approach attempts to circumvent model drift and the need for a time-varying bias correction. The models are initialized by adding observed anomalies to an estimate of the model mean climate. The mean model climate is subsequently subtracted from the predictions to obtain forecast anomalies. Sampling error in the estimation of the mean climatology affects the success of this approach. This is also the case for full-field initialization, although as anomaly initialization is affected to a smaller degree by the drift, the sampling error is assumed to be smaller. The relative merits of anomaly versus full initialization are being quantified, although no initialization method was found to be definitely better in terms of forecast quality. Another less widely explored alternative is dynamic bias correction in which multi-year monthly mean analysis increments are added during the integration of the ocean model.

What I would like to see is independent testing of the models, independent testing on conditions they have not been adjusted to match. The kind of independent testing we expose products to whenever it is of crucial importance that the product performs in accordance with a specification or within certain limits.
Advertisements

One thought on “Model biases can be largely removed using empirical techniques a posteriori!

  1. Pingback: Posts by: “Science or Fiction?” | Science or fiction?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s