Wednesday, October 31, 2012

I am convinced that probability is to our brains what junk food is to our bodies. OUR MINDS DON'T SEEM TO METABOLIZE PROBABILITY VERY WELL.

Hmmph! Where do I begin?

To begin with, there is this ridiculous case where a local Italian court convicted 6 scientists for failing to accurately predict a massive 6.3M earthquake that claimed over 300 lives. 

This case is of particular interest to me because I work in the field of Earthquake risk. No one in the field of earthquake engineering can predict the exact moment when the earthquake will occur. The best science can do is try to guess what is the worst-case scenario earthquake that is likely to occur in a given time (in years). Something like "there is a good chance that we can have a 6.0M earthquake sometime in next 50 years". That earthquake can hit the city tomorrow or 49 years later or may not happen at all, but there is a good chance it can happen within 50 years. These predictions are called "Seismic Hazards" and today, we can make these kind of predictions on every point on planet earth. That's the maximum scientists can do. And trust me, it’s a pretty huge deal that we've gotten this far. After that, the best defense against most of the extreme disasters is adequate preparedness like constructing stronger buildings, station response teams or do what rats do - spend all your life in the damp basement !!!

To be fair, Italy might have inspired The Godfather movies and spawned the ever-awesome Monica Bellucci but to hire some scientists to ask and predict the precise time-window of an impending earthquake and blame them for our helplessness in the face of nature's rage ? Not cool, Italy. Not Cool. After all, not everything in this world is so certain like getting your ass handed on a platter by a gangster because you winked and whistled at his mistress while he was standing right next to her. A lot of times, we cannot be so sure of what's going to happen.

This is mommy nature we are dealing with, Goddammit. How about a little humility in face of nature's ferociousness in form of earthquakes … or tsunamis  … or other natural disasters like when your girlfriend finds out you lied to her to simply avoid spending a full day with her at the mall as she window-shops. Huh? Don't you think we should politely admit our ignorance when it comes to complex natural processes like geo-tectonic movements … or ocean currents … or women?

As if that's not bad enough, I saw the Freakonomics blog trying to scare the scientists by asking:   

Easy, smarty-pants. Easy. Not to pick particularly upon the freakonomics post, but we often expect "perfect knowledge" from scientists even when they deal with uncertainty. It's a common mis-understanding of the concept of probability and prediction itself. Now, the future, stating the obvious, is called "uncertain" and "unknown" for a reason: we are not supposed to know about it completely. Otherwise, what's the whole point? In the freakonomics blog post, he points out a "folly of prediction" that goes like:

"For example: I could write here today that the Dow will reach 35,000 by the end of 2013. In the very unlikely event that this prediction comes true, I would be lauded and rewarded to no end. And in the likely event my prediction doesn’t come true, it will be pretty much forgotten."

The "folly of prediction" means I can crown myself an expert and keep cranking out predictions about the world by consulting a zoo-chimpanzee after over-feed him with bananas. And when I get it right, the world rewards me. When I get it wrong, the world turns to the other drunk chimpanzee who nailed it this one time forgetting all about me. Of course, this is  not fair because there is no penalty for wrong predictions. To keep it fair, why not penalize bad predictions? Well, today we have that "Romanian witches could face jail if predictions don't come true". So he got what he wanted. Didn't he? Now, lets extend the same logic towards the scientists. Yes, why not punish the scientists so that they will learn not to screw up, be careful, work a little harder and better start predicting the earthquakes correctly ? yeah, why don't the damn scientists pay attention to work so that we can be lazy and respond at the last minute or as we call it, "business as usual".

It all sounds like a good idea except that there is one problem: A SCIENTIST DOESN'T WORK LIKE A FORTUNE-TELLER.
The most common problem in understanding risk for us is due to the fact that we tend to interpret what scientists say and what prophets say in a similar fashion. In the interest of public, let me outline a few differences between prophets and scientists:


A prophet delivers a "prophecy".
A scientist delivers a "prediction".

(Notice that I defined the words "prophecy" and "prediction" to have different technical meanings)

Example prophesy: "Event X will happen". Only one possibility for future. 

Example Prediction
: "Event X with happen with a probability of 60%". Multiple possibilities for the future. This number indicating probability (60% etc.) is the most important part of the sentence that tells us what is the likelihood of the event. Prophets needn't come up with that number.


After we know what happenned to event X  …

A prophet would be either right or wrong. Depending on the stakes involved, he must be worshipped or strangled.

The probabilities predicted by the scientist before Event X will have no meaning. Whatever happened happened. The prior probability reflects our level of knowledge about what's gonna happen. A scientist re-calibrates his models with this new data.

The world of probabilities is based on a humble assumption: everything in future is hazy, nothing is certain and every prediction about the future is a lie. The higher the probability of an event, the lesser the chance that I am lying when I say that the event is going to happen. In a strict probabilistic sense, the idea of being absolutely sure about the future is meaningless.  

If someone says, "it rained yesterday", that's a fact. It is verifiable. It's either true or false. But if someone says, "it's going to rain tomorrow", it's always a hunch. You could ask a parrot or a sophisticated computer model based on atmospheric physics but it's still going to be a hunch. There are good hunches and there are bad hunches. Good hunches are based on the scientific process.  If a hunch becomes so good, it's called a "theory". Hunches based on deterministic methods can make very accurate predictions. Like the theory of Newtonian mechanics can predict exactly that what's going to happen to a ball you throw upwards or the theory of commonsense can predict that if you have the habit of falling asleep in a public library, sooner or later someone is going to steal your laptop. A good hunch is validated by lots of data. On the other hand, all bad hunches have a single source: somebody's imagination.


With this background, let's analyze if it's acceptable public etiquette in a rational society to point fingers at scientists for getting things wrong. Going back to the freakonomics' "folly of prediction":
I could write here today that the Dow will reach 35,000 by the end of 2013. In the very unlikely event that this prediction comes true, I would be lauded and rewarded to no end.
First of all, a statement like "Dow will reach 35,000 by the end of 2013" is NOT a scientific statement to begin with since there is no information on what is the probability of occurrence for the event. Hence, it's just another prophetic statement and must be treated as such. A scientist goes through a detailed set of calculations using observed data to arrive at a best estimate for the likelihood that "Dow will reach 35,000 by the end of 2013". If there are no calculations; if there are no probabilities, we all might as well ask the northern spotted owl or a religious book. Theoretically, one could predict 36000 DOW or emergence of aliens that gobble the AIDS virus thereby curing humans of the disease but the probability associated with such a wacky prediction is going to be very, very low for all reasonably logical models. So, what if someone predicts a 36000 DOW with an extremely low-probability and it actually happens? He is still considered a lucky monkey who is good at darts. Or may be he got a damn good model with him, then let's take a serious look at his logic. That's how science works and that's how scientists are rewarded.

Unless the probability of an event is 0% or 100%, there is always a chance that the "most-expected" outcome may not happen. Suppose scientists predict that there is 95% chance that an earthquake may not happen. But that still means there is a 5% chance that it can. And if it does, the scientist is NOT wrong because (technically) he did consider the possibility. That's the beauty of probabilistic analysis, it has the capability to warn us. The scientist or the statistician cannot be blamed for something that beat the odds just so you can feel better. Now I know what some of you are going to say: "Well … yeah, I am not still sure what's gonna happen. So what's the point ?". EXACTLY. Science is not about making you feel better. It's about maintaining a sane level of skepticism about ourselves and our knowledge.

However, we often blame scientists. Perhaps we mistake them as prophets. It's our fault. Also, the scientists' fault if they are not communicating properly. Let's face it, folks. Uncertainty kills us. Who wants to be unsure, doubtful, indecisive, tensed and insecure? Our brains are terrible at understanding any other probability values other than 0% or 100% which means we have a hard time dealing with anything uncertain. Considering the infinitude of rational numbers falling between 0 to 100 and the fact that we only manage to understand just 2 of them, that's pretty, pretty, pretty bad. Moreover, ignoring or misinterpreting the probabilities in the statement "Event X with happen with a probability of 60%" makes it no different from a statement by a prophet. By holding innocent scientists to the same moral standard as a prophet, by demanding absolute certainty in predictions, we defeat the whole purpose of scientific treatment for uncertainty. Hence, the "folly of predictions".

3. WE (probably?) DID THIS TO OURSELVES 

At the root of all this is, of course, our emotional attachment with the future. Our relationship with our future has always been somewhat similar to the relationship between Robert De Niro and Sharon Stone in the movie Casino in that he loves her deeply and she is a selfish shopaholic hooker who doesn't give a damn about him. Similarly, humans are in love with their future but the future does what it wants to do. Thus, we are angry or ecstatic depending on how our expectations are met. It is not surprising that this irrational bonding with the future created a market for prophecies.

I argue that prophecy had to be older profession than prostitution. Long before the men could figure out that they could sell women to their male-friends, we had to figure out first how to keep staying alive. Thrown into wilderness without a guide-book, constantly battling for survival, long before we even invented the numbers, we needed to guess what's going to happen: Will we have food? … Will it rain? … etc. There was a demand for anyone who could tell what's going to happen. To meet this market demand, some brilliant mind would have invented "guess-work" and "comfort-talk" to allay the anxiety of his insecure friends who keep pissing him off with their insecurities. May be he tried rational arguments at first. May be he even told them to be detached and that results don't matter as much as the effort. But we wouldn't listen to all that qualitative, vague, probabilistic talk. We wanted confidence. We wanted guarantees. We wanted to hear what we liked to hear. We said, "Never mind the hazy truth. Just tell me how to get what I want". And we got it. He sold us that Optimism Gospel that "it's all going to be fine" in exchange for food, shelter and even access to his favorite women in the tribe. We were happy with that. I won't be surprised if it's the same crooked genius who monopolized the franchises of bullshitting and prostitution.

The future revisions of the Optimism Gospel was soon recorded officially as religious texts that took a lot of creative liberty in order to explain a lot of existential questions with unapologetic certainty. Religious and prophetic seers have indoctrinated our collective imagination with such notions of absolute determinism. Ever since, we are locked in a system of a binary view of the future that today no one will pay an idealistic astrologer who insists on telling the truth like, "there is a 88% chance that you can get married this year".

Today,  many different faces for prophecy have evolved: religion, horoscopes, tarot cards, crystal balls, talking to souls, invoking spirits, parrot fortune telling, psychics, graphology, palmistry, personal portfolio managers, hedge funds,  algorithmic trading and derivative markets. It must be noted that ancient forms of prophecy are more entertaining than modern-age versions of prophecy industry which involve an MBA degree in finance.

NOTE 1. The case is not just about risk prediction. The unfortunate aftermath of the earthquake is more a result of bad risk communication which had to do with government officials other than scientists. As with anything human, there were lot of personal politics involved between scientists and local government officials. For more details, refer to [link]


Coming soon in this series:

1. Why can't we predict complex phenomenon like earthquakes, climate-change or your own career trajectory so easily?

2. How to use the probability for evil-but-profitable purposes?



Twilight Musings said...

Thanks for writing this. I am extremely mad to made any sense to non-expert audience if I write about this.

A couple of points: Part of the problem with Italian verdict might be their scientific system and how research scientists and bureaucracy interacts. I don't know but I get a sense that there is this underlying 'trust' issue.

Second: I hated the way Lalliana Mualchin (in Scientific American article in Feb 16 2012), throws Italian Scientists under the bus to argue that DSHA is better than PSHA. As if low hazard estimate by PSHA is the real reason and not unreinforced masonry walls that are a threat to life safety.

I think all scientists should tattoo: Risk = Hazard X Consequence, on their foreheads.

Also, prophecy is supposedly a female persuasion. Remember Oracles of Delphi? or 'Aakaasa Vaani'. (I don't know why it is always translated with feminine gender)

Some might wonder what is the use of probability at all. But, how are people using "probability of precipitation" ((PoP) in everyday weather news? If it is 10% and I have nothing important planned, I don't care for a rain jacket. OTOH, it was an important meeting day, I will take one (as the consequence of getting drenched isn't small anymore). Aren't people making these kind of decisions everyday?

Twilight Musings said...

Btw, here is where ex-chief California DOT seismologist rises my ire:

Sash! said...

@twilight: Thanks for reading it.

I was irate too. I had to temper down a lot of misplaced sarcasm. I was screaming "DON'T YOU GET IT? ISN'T THAT OBVIOUS? HE IS A HUMAN TOO. NOT A GOD OR AN ASTROLOGER WITH EXCELLENT COLD-READING SKILLS"

I don't know .. may be we don't teach probability that well. May be do but our mind doesn't pay attention unless proper stakes are involved. Either way, we suck at it. Yeah ... our decisions are intertwined with our expected pay-offs all the time.

Yeah. Actually none of the scientists explicitly said in the report that "it's perfectly safe". That was what the government official said to calm people down and the very scientists disagreed with his released statement. Check that link I provided.

I just read that Lalliana Mualchin's piece in SciAm. Gosh, he even testified !!! What an a**hole! What else can I say? It's about turning a perfectly probabilistic phenomena into a something like a dogmatic, religious prophecy and blaming humans for not being God.

Hindsight is a wonderful thing. Everything seems like all disasters could have been avoided and all opportunities should have been availed. Ahaa .. that classic state of "maya".

I think risk engineering is dirty business. No one notices if you do a good job and nothing bad happens. When something bad happens, we all march down the "if only ... " lane, to point fingers at you for not doing a good job.

I never observed that. Regarding feminine flavor to the prophecy. Interesting ...

Twilight Musings said...

Yeah. I read the link to Science blogs.

But all this discussion basically ignores that people survived even bigger earthquakes around the world.

If someone must be punished for manslaughter it should be the ones who approved unsafe buildings in an active seismic zone.

The fatalities happened due to bad buildings (unreinforced masonry) not bad prediction of a single earthquake occurrence.

Sash! said...


Post a Comment

Go Ahead. Blurt it out ...


Copyright 2010 F L A W S O P H Y .