Science writers at The Economist explore one of the known problems with climate modeling:
Psychologically, people tend to be Bayesian—to the extent of often making false connections. And that risk of false connection is why scientists like Pascal’s version of the world. It appears to be objective. But when models are built, it is almost impossible to avoid including Bayesian-style prior assumptions in them. By failing to acknowledge that, model builders risk making serious mistakes.
The layman-level example they provide shows just how fundamental an issue this is.
Fascinating article. Reading it now, thanks.
On a personal note, what’s humourous to me, is that I passed my Stats final in college by guessing the answer to a question on Bayes Theorem. Hmm not sure if you could classify that answer as a Bayesian response based on prior knowledge and assumptions or a Pascalian roll of the dice.
I ran into this problem a lot while working with economic models built from econometric studies. The damn models would behave in insanely weird ways depending on the initial values and variables used/excluded, and these were always the subject of debate and interpretation. And if you thought that was fun, you could always try “tweaking” the mathematical relationships between the variables, and then run for the hills while the model blew up in your face.
This is why I simply can’t get excited when some multi-hundred- (or worse, -thousand)-simultaneous-equations climate model referenced by the IPCC for their latest “forecast” of climate doom tells us that we’re all gonna die horribly via climate change. My cynical reaction is “you folks oughta change some of your input assumptions and behaviour parameters — you’ll have much more optimistic outcomes.” Meanwhile, it behooves real scientists to continue to study and learn about the potential issues, rather than pre-judge based on someone else’s prejudices and assumptions about the existing state of the world.
Computer modeling is still in its infancy, and has problems correctly specifying all the relevant variables and interrelationships even in deterministic systems. It’s much worse when dealing with mathematically chaotic, non-deterministic physical systems (which climate certainly is).
garth – many thanks; excellent comments.
Like usual, popular science reporting is painful to read…
First, the author sideswipes the Bayesian approach: “Psychologically, people tend to be Bayesian—to the extent of often making false connections. And that risk of false connection is why scientists like Pascal’s version of the world.”
Lazy scientists (not a slur) like Pascal’s version, but good scientists use it only for cases where the sample population is a good approximation to random. For example, the sun will rise tomorrow, or it won’t – two choices, thus Pascal’s approach (improperly) assigns a 50-50 probability to each. Bayes says that we have so much prior information about this system (the sun having risen very many times before), the probability of it rising tomorrow approaches 100%. In other words, we are not justified in predicting the opposite without some very compelling NEW information.
It is thus clear that many climate modellers do NOT fully implement Bayesian stats, because they treat their weak predictions with too great a seriousness.
Once more to the article, and blowing big-time the layman-level example”:
“The particular range of values chosen for a parameter is an example of a Bayesian prior assumption, since it is derived from actual experience of how the climate behaves—and may thus be modified in the light of experience.”
This is basically true, but incomplete. I’ll get to it.
“But the way you pick the individual values to plug into the model can cause trouble.
They might, for example, be assumed to be evenly spaced, say 1,2,3,4. But in the example of snow retention, evenly spacing both rate-of-fall and rate-of-residence-in-the-clouds values will give different distributions of result. That is because the second parameter is actually the reciprocal of the first. To make the two match, value for value, you would need, in the second case, to count 1, ½, ⅓, ¼—which is not evenly spaced. If you use evenly spaced values instead, the two models’ outcomes will cluster differently.”
Aargh! Gobbledygook! A proper statement would close with ‘Since many climate modellers make stupid mistakes in their attempts to do a sophomoric implementation of what is a standard statistical technique, one should not put too great a faith in the results of the simulations.”
Now, to be fair, in climate modeling the parameters are spaghetti-coupled and highly incestuous. Here, then, is where the power of Bayesian approach comes into play. One can guess at these couplings, iterate, look for self-consistency and clustering, and new information, do stability analysis…and come up with a pretty decent answer. Bayesian statistical analysis is iterative, and the prior probabilities are incrementally modified based on such an analysis.
I doubt that this is the majority approach climate modelling.
Anything so involved has to be given a title. Thus: “Science”.
This magic word, applied to whatever one chooses, can not be denied by the unfortunate mental processes we are taught to use.
Therefore so many hoaxes work (especially on the internet) so that we might prove the existence or non-existence of any subject that rocks our boat.
My nomination for the absolute worst: TV “News”.
People also, consiously or unconsiously, will project what they want or expect to see into the model they build. If you want to or expect to find that Carbon Dioxide is causing a heeat build up then the model you build will show exactly that.
I
Global warming is cuased by the hot air from liberal politicians and enviromentalists wackos
All computer models have a very simple formula that has never failed to work:
Crap in = Crap out
The soviet system didn’t work because their economist planners didn’t get their calculations right on their 5 year plans. The economy is not as difficult as the natural world. Its relationships are infinitely more complicated.
Only leftards and other morons think you can model the entire planet and get it right over 50 or 100 years.
Now I know why I come here. I’ve learned more in one year from ET,Viturius,tenebris and others then I could in 6 years of university. Yes. Even the leftards have taught me a bit. I still like the old proven computer formula that Warwick gives. Now THAT is settled science!
Tenebris, you seem a proponent of Bayesian analysis and obviously are much more knowledgeable on this topic than I am.
What do you say to this criticism, would you agree or disagree? Is Bayesian analysis fundamentally flawed?
“For over 150 years, Bayes’s Theorem formed the foundation of statistical science, allowing researchers to assess the meaning of new results. But during the early part of this century, a number of influential mathematicians and philosophers began to raise objections to Bayes’s Theorem. The most damning was also the simplest: different people could use Bayes’s Theorem and get different results.
Faced with the same experimental evidence for, say, ESP, true believers could use Bayes’s Theorem to claim that the new results implied that telepathy is almost certainly real. Skeptics, in contrast, could use Bayes’s Theorem to insist they were still not convinced.
Both views are possible because Bayes’s Theorem shows only how to alter one’s prior level of belief – and different people can start out with different opinions.
To non-scientists, this may not seem like an egregious failing at all: what one person sees as convincing evidence may obviously fail to impress others. No matter: the fact that Bayes’s Theorem could lead different people to different conclusions led to its being inextricably linked to the most rebarbative concept known to scientists: subjectivity.
Link: http://tinyurl.com/2jngvr
The Paper is by a Robert Mathews- his site is here. http://www.robertmatthews.org/
Nice article, even better comments. Thanks Garth and Tenebris. Attempts to model the monsoon, arguably the most significant weather pattern on the planet, have gone nowhere; what more imbecilic claims to have modelled the entire earth’s climate 100 years into the future?
Hey, they predicted a very busy hurricane season last year, and……wrong again. But I saw last night they’re predicting the same busier than normal activity this year….50/50 shot at it again…
JCL
This article is absolutely correct. And we should all be demanding that the “climate scientists”, most of whom are paid for by your and my tax dollars, must release all of their calculations so that we (and especially the Steve McIntyres of theworld) can evaluate and probe the assumptions and calculations that are used to generate the apocalyptic scenarios. If not, the “settled science” being foist upon us will most likely end up being proven wrong and all of science will collaterally suffer a lack of credibility. Where are the other scientists in fields which still value the scientific method?
“Scientists Confirm Long-held Theory About Source of Sunshine.”
http://www.rushlimbaugh.com/home/daily/site_082207/content/01125112.guest.html
boze:
Bang-on about the source of funding and the prejudicial viewpoints that spur out from the assumptions of purity.
Seems to be if the funding comes from taxpayers, the results are purer than those that come from funding by evil corporations.
As I’ve posted before, if the science is settled, perhaps it’s time t cut off public fundings, since there is nothing left to learn.
Hmmm. Maybe we are in the end times.
My wife keeps asking me what is all this “settled science” hooha? It’s not a legal case fought between lawyers (although some people are attempting to do that) like it was a court case per se. Science is , well a theorem or hypothesis that other scientists try to replicate using the originator’s data or new data added to the mix. SO, when did the science become “settled”? Was a court case held and a ruling made?
We might get to that court case (I am REALLY looking forward to that!) this fall when the four amigos, De-yawn, TJ, Gilles and Mayo-naive take the Cons to court over their stance on KYOTO. I say let’s have at ‘er, ’cause then we can present all sides in the big debate and get to see all the “science” behind this (in my view) fraud and use all the legal muscle of calling witnesses and data sets to make our own minds up about this. As a scientist, of course I’m biased. I believe in the scientific method, because it has been working for a long time. I posted a link yesterday that didn’t get much play. It’s quite a simple paper with lots of colorful graphs and charts that help the writer make his presentation very effectively. It speaks to the level of our politicians and the unscientific locals, so that even they can form their own conclusions. I call it GW for Dummies.
http://www.lavoisier.com.au/papers/Conf2007/Archibald2007.pdf
I have always posted in the past that GW is actually beneficial to this NORTHERN country. Warmer winters = longer growing seasons = more food produced here and the whole sytem of agricuture can benefit from INCREASED CO2. It’s only a scientific fact that has been replicated in the lab.
Tenebris, it’s been awhile since I cracked a book on this so bear with me. Do you feel this is a legitimate criticism of Bayesian Analysis, that at its heart may rest a fundamental flaw?
“For over 150 years, Bayes’s Theorem formed the foundation of statistical science, allowing researchers to assess the meaning of new results. But during the early part of this century, a number of influential mathematicians and philosophers began to raise objections to Bayes’s Theorem. The most damning was also the simplest: different people could use Bayes’s Theorem and get different results.
Faced with the same experimental evidence for, say, ESP, true believers could use Bayes’s Theorem to claim that the new results implied that telepathy is almost certainly real. Skeptics, in contrast, could use Bayes’s Theorem to insist they were still not convinced.
Both views are possible because Bayes’s Theorem shows only how to alter one’s prior level of belief – and different people can start out with different opinions.
To non-scientists, this may not seem like an egregious failing at all: what one person sees as convincing evidence may obviously fail to impress others. No matter: the fact that Bayes’s Theorem could lead different people to different conclusions led to its being inextricably linked to the most rebarbative concept known to scientists: subjectivity.”
Here is the link:
http://tinyurl.com/2jngvr
In short can any climate model be considered valid unless all parties agree to a standard level of “belief”?
I took a “bird” course in university called Statistical Analysis and Graphical Presentation. In effect a course on: How to Cook Your Data to Make (not prove) Your Point. I aced it. Statistics is a wonderful TOOL, so is politics.
The only problem with climate modelling is that some people, and unfortunately some influential people at that, take climate models seriously. Constructing and running such models is scientifically interesting, primarily as showing where our lack of knowledge is deepest. In another hundred or two hundred years we may know enough about modelling complex ecosystems that we could actually begin to trust our models.
Mathmatical formula. Number of climate change variables approaching Infinity. Amount of human uderstanding of interaction of said variables approaching Zero. Human ability to express said understandings in mathmatical formula absolute zero. Therefore here is a mathmatical formula that covers the bases.
AGW = JS
Where JS is understood to mean Junk Science.
Blazingcatfur ….. when in doubt go with Pascal.
eh?
Blaise Pascal – what a name
But when faced with a true dilemma – Bertie is my boy!!
😉
all models are theory. based on what? whatever the person creating the model wishes.
@blazingcatfur…
By my reading, the article you quoted is sympathetic to Bayesian stats, noting the recent (but erratic) move to critique P-value tests. This critique is the subject of his article, but his narrative style is such that he gives the impression by mid-article (where your quotes come from) that the Bayesian approach is fundamentally flawed.
To answer your question: No, the Bayesian approach is not flawed…says the heretic :-). The strength of the approach is that it “shows only how to alter one’s prior level of belief”. That means that, eventually, no matter what the starting belief (assignment of probabilities), everyone will eventually come to the same conclusion (in theory, assuming convergence).
The “problem” with the Bayesian approach is that it is tedious to implement. It requires care and patience as one tests the results many times to “generate” robust/stable “beliefs”.
I look at global warming this way.
Hurricanes and monsoons occur almost every year.
The impact of them varies year to year.
Now if the all the learned scientists, armed with huge amounts of real and current data about them, cannot get something as simple as monsoon and hurricane predictions right, why in the hell would you ever believe them in predicting the earth’s weather over the next 10 or 50 or 100 years?
They cannot get the hurricane predictions right, or sometimes even close (like 2005/2006), because there is a little item, or process, that they clearly do not understand about how the ‘heartbeat’ of the earth works.
I still have to laugh at the people who think that man’s manipulation of miniscule amounts of greenhouse gases is going to control the climate.
They seem to have lost track of the scale of the thermodynamics involved.
OMMAG – I refuse to rise to the “bertie” bait. While I grant the man was a better mathematician than me (duh!), I think he was led astray in some aspects of it by his philosophy (never listen to Wittgenstein). I accept neither his philosophy nor his example of life.
The punishment for denial of Bayes’ Theorum, which is effectively illegal in western civilization, should be death.
Sorry, but I’ve seen one too many discussions of crime in Toronto screwed up by the inevitable jackass who offers “Not all Jamaicans commit crime, just like not all white men are serial killers”, to the point where I consider Bayesian deniers to be a clear and present danger to national security.
“Statistically-uncertain afterlife” to the Bayesian deniers!
…garbage in, garbage out. The difficulty comes when you do not know what garbage looks like
Priceless.
I see what you did there, Tenny, and might have to appropriate it. Mimicry, flattery, etc.
Look at the previous post where health officials claim to be genuinely stumped as to why Saskatchewan has high drug abuse rates. What a perfect example of Bayesian denial.
Many, many years ago, I ran into a high school colleague class-mate when he was doing his PhD. He was always good at math, and, apart from that one fight we had in the school grounds one day, we got on fine, both being mathematically inclined and liking chess.
But I digress.
He was modelling air flow over the Andes!!
Good luck I thought.
Now this innocent academic mathematical pass-time has become a BIG INDUSTRY funded by BIG GOVERNMENT. The models have become seriously important to the financial wellfare of thousands of such people.
To sustain them, we must have data to input and the better the data, a priori, supports the output of the models (Bayesian), the more modelling there will be. Hence the importance of inputting rising temperatures (hockey stick anyone?) to these models that predict rising temperatures.
The real whores in all this are not Flannery, Suzuki, Stern and Gore, but the producers and manipulators od data for input to these models; because, seeing the real world data, they know it is false. I am talking of Hansen, Mann et al.
Who cares about models…
Global warming, at least in my neck of the woods (N. Ontario), has been very real in the last 25 years. Maybe I should call it local warming 🙂
Back in the early 80’s I recall needing 6-8 full cords of firewood to go through the winter. Now 4-5 are more than enough, even after adding 300 square feet to my house. Less work for me.
We grow our own food. Our growing season is 4-5 weeks longer. Less work again.
Our lake used to be frozen till the first week of May. Now it’s ice-free by mid-April.
To this guy, warming has been very real, and a good thing.
Now this fellow is predicting colder climate is around the corner:
http://www.lavoisier.com.au/papers/Conf2007/Archibald2007.pdf
Say it ain’t so.
Concerning Bertie Russell: I had seen him as a kid on TV in anti bad-things demonstrations and marches.
Then, later in life, I came a cross one of his books and thought: “Hey, this guy thinks like me!”
But I both respect and dispise him as he is both very right and so very, very, wrong at times. And, also, a pompous arse-hat.
Clarification:
…I had seen him, as a kid, on TV
Goodonya GreenNeck: Holocaust deniers; moon landing deniers; Bush is dumb deniers; global warming deniers–the list goes on and on. Obviously very few people on this blog live in northern Canada where the effects of global warming are real–and devastating to a noble way of life.
Tenebris,
Is handicapping the ponies an example of Bayesian behaviour? These people certainly are not punting on a 1/8, 1/10, etc chance of winning.
Given that they are betting their own cash, it probably in their best interests to win (whatever bet they make). They are given all sorts of history on the horse and jockey.
When we look at the results tables pick any one:
http://www.drf.com/results/rindex.html
It becomes apparent that very few of the punter’s favourites win … a favourite (most money on it) will have the smallest payout … I’ll generalize a bit, favourites typically have payouts of $2 and change to, maybe $5. If you bet the favourites all the time you would lose your shirt.
This is just for x amount of horses, in a race, on a given day, etc. Should be easier to model than our climate system.
I saying that just because a horse beat another horse 5 times in a row, doesn’t mean he beats him the 6th time. I’m not trying to be argumentative … just looking for your comments.
you-conn
“where the effects of global warming are real–and devastating to a noble way of life.”
could it possibly be regional (hemispherical) warming????
and greenNech wuz praising it’s benifits
@blazingcatfur
further to your question. As Tenebris says everyone’s posterior beliefs will eventually converge at the same answer given enough information and time. For climate change modelling this might take decades and we want some certainty about answers now. One way of handling the Bayesian approach in scientific analysis is for the researcher to be explicit about all prior probabilities and alter some priors to show the sensitivity of results to the assumptions. Model builders have to build in some prior assumptions to make models operable and the advantage of the Bayesian approach is that it makes those assumptions explicit.
While on the topic of probabilities, one of my beefs with the climate change brouhaha is the rare recognition that we are discussing a series of conditional probabilities. For instance, the probability of my statement that we have global warming because of man-made co2 and that the implementation of Kyoto will help the world is based on the probability that global warming is occurring, p(A), given that there is warming the probability that it is caused by co2, p(B|A), the probability it is man-made given that warming is occurring due to co2, p(C|A,B) and the probability Kyoto will help conditional on man-made global warming, p(D|A,B,C). The probability of my statement, p(A,B,C,D), i.e., all four events are true, is p(A)p(B|A)p(C|A,B)p(D|A,B,C). If I assign nine chances out of ten for the first three statements being right and eight chances out of ten (.8) for p(D|A,B,C), then there is less than six chances out of ten (.58) of my statement being right. Much of the argument around climate change doesn’t recognize the cumulative uncertainty.
Yukon Jack – Don’t worry, very soon the temp will go down .6C and you then can go enjoy your noble life again.
PS. You’ll need something finer than a Sharpie to mark your thermometer for your “un-noble” and noble level.
ural
Consistently betting the favourite, the long-shot or a middle odds horse will cost the bettor his shirt in the long run because of the track take. Every race the payout is less than the amount bet so as a group punters lose every time. Maybe Bayesians don’t bet because they’ve learned this fact the hard way since study after study has shown that in horse racing the long-shots are overbet and favourites underbet. In other words, long-shots win less often than their post-time odds and favourites win more often than their odds, i.e., favourites should have even lower odds and payouts. Clearly, punters are not Bayesian. The financial economics academic literature has many of this type of study.
@ Tenebris & AltaInd, thank you both for the assist. I remember now why I dreaded stats class;)
Bayesian Statistics – or at the minimum, the application of Bayes’ Rule – are applicable & accurate in a wide variety of “real life” situations, from the diagnosis of disease based on test criteria, the programming of anti-spam filters, and (in my industry) the prediction of automated electronic bottle inspector (EBI) accuracy when rejecting production bottles.
Part of the problem folks have with Bayes is the formula (& derivation) for Bayes’ Rule, and the counter-intuitive “feel” you get when using a priori data when predicting statistical events. This latter point tends to drive non-Bayesians somewhat crazy.
Want an example of how an educated person may be confounded by Bayesian Statistics? OK. Next time you see your MD, ask the following: “A person is tested for a disease that affects 1% of the population. The test will give a true result if the person has the disease 95% of the time, but also gives a false positive result if the person doesn’t have the disease 0.5% of the time. What is the probability the person actually has the disease if they test positive?” Most folks (MD’s included) will say, “95%”, which is wrong. The correct answer is 66%, or there is a 1 in 3 chance the person DOESN’T have the disease, despite the accuracy of the test! [Anybody interested in the math can email me for details]
Some time ago I used Bayes’ Rule & the last 4-5 years of accumulated Metro Toronto Police R.I.D.E. data (and a “reasonable” estimate of roadside machine accuracy) to calculate the probability that a random “positive” handheld breathalyzer result actually meant the person was impaired. As I recall, the margin for error was so great, arresting and prosecuting somebody with ONLY the handheld test & no other verification testing could be grounds for “reasonable” doubt and hence acquittal, if the accused had a Bayes-savvy lawyer.
I’m not certain how these scientists would use Bayesian Stats for climate prediction, although I suspect it will involve more advanced analysis – Bayesian Belief Networks, likely – which use Bayes’ Rule but can get considerably more complicated, depending on the model detail.
mhb23re
at gmail d0t calm
I have experience with sophisticated 3D CAD models. It’s GIGO.
“Computer climate modeling” to me sounds like people playing SimCity/SimUniverse and enabling Random Disasters in the game’s settings:
“Hmmm, this is boring. Not enough happening fast enough, I need to add some drama to my little self-created world. Let’s see what happens if I add unexpected Godzilla attacks, big-ass meteorite strikes and global warming. That ought to be exciting!”
Oops, I forgot to add, in response to:
In one sense it is obvious that assumptions will affect outcomes…
No fair! You changed the results simply by observing the experiment!
Oops, I forgot to add, in response to:
In one sense it is obvious that assumptions will affect outcomes…
No fair! You changed the results simply by observing the experiment!
DAMN. I always thought this modeling thing was something with no brains no tits and wore funny clothes. Are you telling me now that it’s something with lots O brains no clothes and no wits?
“Climate modeling” = garbage in garbage out.
“vatiables” = wild cards
Climate science is as precise as practicing archery in a blizzard
“Climate modeling” = garbage in garbage out.
“variables” = wild cards
Climate science is as precise as practicing archery in a blizzard
For the record, Yukon Gold is in no way related to Yukon Jack. Thanks.
Re rockyt’s post: inability to predict does not necessarily imply lack of understanding. It is possible – easy, even – to set up a model about which everything is known, and which allows of no predictions beyond the immediate. This is known as “deterministic chaos”, and James Gleick wrote a fairly good book about it. The “logistic map” is a good example. Wikipedia has a good article on the logistic map, BTW. Simple equations for weather (Lorenz model) have solutions with this property. So quite possibly the conclusion of climatologists two hundred years in the future, if there are such people, will be that climate cannot be modelled for more than a month or so into the future, beyond the changes due to changes in sunshine received due to changes in the Earth’s orbit. It may well be that ALL that can be known about our climate is that it can’t be predicted, even with a vast amount of input data and well checked evolution algorithms.
And aren’t we about 2000 years overdue for the onset of the next Ice Age?