Communicating uncertainty - a cautionary tale

Communicating uncertainty, via the media, is very difficult. Just ask the Met Office, which has found itself in something of a storm after revising its forecast of this summer's weather. And it's perhaps a cautionary tale for the rest of us.

Back in April, a Met Office press release stated we were "odds on for a barbecue summer". But after a couple of weeks of widespread rain, and grumbling among those who chose to go on holiday in the UK this year, the popular press have scented blood in the water.

And so the Met Office has been forced to backtrack, with its Chief Meterologist appearing on TV news all over the place to explain that "odds on" actually meant a 65 percent chance. So there was a 35 percent chance of another wet summer like those of 2007 and 2008.

Is there anything we can learn from this? Here are a few observations:

(1) Nowhere in the Met Office press release from April - which you can read here - is the 65 percent chance mentioned. All we get is "odds on".

Of course, the full details of the prediction are available in the linked technical report, but in this age of churnalism, isn't it a bit optimistic to expect most journalists to download and read it?

Well, despite the lack of actual probabilities in the press release, science correspondent Richard Alleyne at the Telegraph did a good job of digging them out and making them clear in his coverage here.

And here's Roger Harrabin on BBC News, who eventually, in the last moments of the package, mentions the 35 percent chance of a wet summer, despite no mention of it in the press release.

So maybe those science journalists aren't utterly incompetent all the time, eh Ben Goldacre?

(2) The Met Office press release did contain a clear caveat that its long-range predictions are "not forecasts which can be used to plan a summer holiday or inform an outdoor event."

So why did one of the Met Office's public weather service managers say to the Daily Telegraph that "she hoped it would encourage Britons not to travel and to boost UK tourism by taking their holidays in the country this summer"?

(Eagle-eyed readers will have spotted that the same spokesperson also popped up in Roger Harrabin's BBC piece above, saying the same thing...).

And why does BBC weather presenter Simon King (a Met Office employee, I believe) take great pains to explain what "drier and warmer" means in this clip, but not bother to explain what "odds on" means? And then go on to say "all in all, not looking too bad at all, if you've got any outdoor activities planned"?

Overall, I don't think the current storm has anything to do with the Met Office's ability to forecast. But it does raise concerns over the Met Office's ability to communicate uncertainty.

Here's a quick recap of some of the Royal Society guidelines for communicating risk and uncertainty:

Q: "Can the risk be compared with anything else?"

Answer here: no comparison was made in the press release. But they could, for example, have said that the 35 percent chance of a wet summer was still 4.9 million times greater than the chance of buying a winning ticket in the national lottery. If you put it in that kind of context, maybe folks won't bank on getting the bbq out.

Q: "Have I warned about drawing the wrong conclusions about the risk?"

A: yes, the press release does have the caveat to that effect. But clearly Met Office staff being interviewed by the media were ignoring it.

Q: "Could the reporting of my work lead to undue anxiety or optimism among audiences or readers?"

A: yes - and that's why you're in such a mess!

Perhaps some training for Met Office staff in this area might be in order? I can recommend a great company :)