Long range forecast verification-how are we doing?

Wiseweather long range forecast success

How we calculate our forecast ratings -

We compare each forecast with the actual values presented officially by the Met Office at the end of each month.

We actually use the latest 1971-2000 'CET means' to compare against the temperatures but -like the UKMO- the 1961-90 means for England and Wales sunshine and rainfall. We then award a rating to each element of the forecast, temperatures, rainfall and sunshine. We use the following criteria to award these ratings. For rainfall & sunshine -if over 26% out =poor, 11-25% out =fair, 6-10% out =good, 5% or less out =very good.

For temperatures - over 1.5C out =poor; 0.8-1.4C out =fair; 0.4-0.7C out =good, less than 0.3C out =very good.

For months where extreme value means are recorded this is not necessarily a reliable indicator though of forecast skill & accuracy of course, so care needs to be taken in interpreting and using these figures.

How well?

Temperature

Rainfall

Sunshine

July 2003

FC+0.8C

ACT 1.1C V Good

FC 85% ACT 121% Poor

FC 125% ACT 94% Poor

Aug 2003

FC 1.8CACT+2.1C

V Good

FC 75% ACT 23% Poor

FC 130% ACT 121% Good

Sept 2003

FC +1.5C ACT +0.6C Fair

FC 80% ACT 47% Poor

FC120%ACT 125% V Good

Oct 2003

FC +0.5C ACT -1.2C Poor

FC 125% ACT 67% Poor

FC 100% ACT 135% Poor

Nov 2003

FC 0.0C ACT +1.2C Poor

FC 50% ACT 109% Poor

FC 130% ACT 106% Fair

Dec 2003

FC+0.3C ACT -0.3C Fair

FC 80% ACT 104% Fair

FC125% ACT

117% Good

Jan 2004

FC-0.4C ACT 1.0C Fair

FC 90% ACT 137% Poor

FC 110% ACT 102% Good

Feb 2004

FC+0.8/ACT +1.2 Good

FC 80% ACT 91% Good

FC 125% ACT 135% Good

As can be seen the eight months before March 2004 did fairly well -but not well with regard to rain again.

Temperatures were reasonably well predicted (acceptable 6 out of 8 months), sunshine quite well predicted (acceptable 6 out of the 8 months) but rainfall was again 'off' much of the time, only 2 out of 8 months being acceptable. However general rainfall trends (ie will it be a wetter or drier than average month were right a little more than this but overall admittedly rainfall was not very well forecast.

How well we have done more recently- 2004 through to 2007

Looking at more recent data up we have tried hard to reassess and regauge our methodology to develop the monthly forecasts in this time. We think that this has paid off. Temperature forecasts were 'acceptable' 7 out of the 8 months. Sunshine forecasts 'acceptable' every month. The most difficult area to get right has again been rainfall, we've only had 'acceptable' ratings 4 out of the past 8 months for this. For May we were 44% off, the heavy rain forecast in the first 2 weeks in the south failed to fall, July 26% off (not too bad-though not very good either) and August 80% off.. this was due to the 'unusually wet' August- hard to forecast. October was a wet month, we thought it would be slightly drier than average.

11 out of 24 forecast areas have rated as good or very good and 8 more were rated 'fair' (ranking criteria are stated on the page above) making 19 out of the 24 areas forecast over the 8 month period 'acceptable', in our opinion (80%). This compares with a 'forecast success' rate figure of 14/24 (59%) in the previous eight months.

How well?	Temperature	Rainfall	Sunshine
March 2006	Fair	Very good	Fair
April 2006	Good	Fair	Very good
May 2006	Good	Poor	Very good
June 2006	Very good	Very good	Fair
July 2006	Fair	Good	Fair
August 2006	Good	Poor	Very good
September 2006	Very good	Fair	Very good
October 2006	Good	Poor	Fair