Randomness, risk and uncertainty: How do we know what we don’t know?

Being a habitual sceptic (and an economist), the insights offered by Nassim Nicholas Taleb in The Black Swan have struck a chord. I have never found a receptive audience in academia for my dislike of the assumptions of the characteristics of randomness that determine the probability density function (amongst other assumptions) in most statistical analysis – especially in social phenomenon. But finally I have a wing man.

The general attitude I face is that if we don’t make these (sometime radical) assumptions, we can’t do any analysis of the data, and draw any conclusions. My response is; what use are conclusions based on flawed assumptions?

The book poses the challenge to think rationally about probabilities, and the impact of improbable events. In particular, Taleb challenges us to acknowledge the limits of knowledge. Real risks and randomness come from the unknown unknowns.

He uses an example of casino to explain the difference between known risks that occur in a world he calls mediocrastan, and the wild unknowns and events from the world of extremistan. The mediocristan risks are those involving the gambling itself. Each individual bet has a risk that is essentially Gaussian, so with a large number of bets taking place, and limits on the size of each bet, these risks are eliminated.

Taleb suggests that most of our concerns about risk, and the high impacts of improbable events, are from the world of extremistan, where complex systems result in variations at all scales. The point is that in extremistan, large variations and extreme events WILL HAPPEN, and much more often than we think. Such large complex systems include financial markets, the global economy, and the climate.

While attending a statistical conference at a Las Vegas casino, Taleb discovered that the four largest losses incurred by the casino did not involve the gambling itself (whether cheating or otherwise). The first was when a tiger performing in a stage act maimed one of the performers. The next was when a disgruntled contractor who became injured on job threatened to blow up the casino with dynamite because he was insulted by the settlement offered. The third was when an employee failed to mail paperwork to the Internal Revenue Service for a number of years, which ended in a monstrous fine. And finally, the casino owner’s daughter was kidnapped and held ransom, which forced the owner to dip into casino funds.

These events are Black Swans. Unpredictable, outside the scope of expectations, and have massive consequences.

He makes a number of interesting points that I want to share. These are particularly relevant in current environmental debates. For example, where I work we try to estimate the environmental impacts from changes to stream flow in rivers. The number of assumptions in unbelievable, and any output from this type of modelling has to be taken with a grain of salt. It is merely some background information that either confirms or challenges the experiences on the ground. When I write about the economic impacts of changing water regimes I repeatedly make the point of acknowledging the unknowns and the limitations of my analysis. Can you imagine the complexity of climate models, and the staggering number of assumption built into them? One wonders whether climate scientists understand statistics at all.

The first point of interest may be familiar for those who are statistically inclined. It is the statistical regress argument and it is a cause for concern. It goes as follows:

Say you need past data to discover whether the probability distribution is Gaussian, fractal, or something else. You will need to establish whether you have enough data to back up your claim. How do we know when we have enough data? From the distribution – a distribution tells you whether you have enough data to “build confidence” about what you are inferring. If it is a Gaussian bell curve, then a few points will suffice. And how do you know if the distribution is Gaussian? Well, from the data. So we need the data to tells us what the probability distribution is, and a probability distribution to tell us how much data we need. This causes a severe regress argument.

Given that our data samples for global temperatures are extremely limited, climate scientists face this problem at the outset.

Another interesting point is how the nature of Black Swan events, and the resulting silent evidence distort our interpretation of history. Any act that aims to prevent a Black Swan event goes unnoticed because its success can never be observed. Imagine there is a bureaucrat who decides to implement aviation rules in August 2001 that would have prevented the September 11 events in New York. We could never judge the success of these measures in preventing terrorist attacks, and the bureaucrat would never gained any credit for the measures. Possibly, due to the complexity, cost and frustration of travellers, he would have had to overturn the rules in 2002. He would be labelled as someone whose best skill is to waste time and money. Learning from history is very, very distorted due to silent evidence. I can imagine in the not too distant future that the history book will explain how we should have seen an event like September 11 coming, due to such things as ‘rising tensions between terrorist groups and the US’, but they would simply be wrong. The crashing planes were the sign of rising tension!

Another great insight is the problem of induction. He uses an analogy of a melting ice cube. If we know the shape of the ice-cube, we can fairly well predict the size of the puddle of water when it melts. But, if we have the puddle of water as our source of information, there is not much we can say about the shape of the ice-cube. In economics we constantly go about measuring puddles of water, and through flawed statistics, try and make outrageous claims about the shape of the ice-cube. The herd mentality of the global economics profession and media seem to have induced that overzealous lending in a few sub-prime locations in the US has led the whole world into a massive recession. My question is, the given how many other more significant events were happening around the globe during this time, how can anyone be so sure of that the ice-cube was shaped like a few bad loans, and not like a oil shock? Or why was the cause not simply a unique combination of unforeseeable events? This same question can be applied to climate change. If we agree that the climate is changing (which itself is questionable due to the previous two reasons), how can we isolate a single cause in a complex system?

I will stop now because I don’t really think I can do justice to the ideas of Taleb and his philosophical predecessors here. I just want to reiterate that we know a lot less than we think we know.

My main concern is that for someone who preaches a precautionary approach to making claims of knowledge, Taleb is a devoutly religious man who has used arguments such as ‘religion has not killed so many people as the concept of the nation state’. So, if religion is the root cause of, say, 10 million premature deaths, while fighting for or defending a nation state (which coincidently have often been religious states) has killed, say, 20million people prematurely, does this mean that religion is good for society? Taking this argument elsewhere and we get such things as ‘murder kills 100 people annually, but motor vehicle accidents kill 300 people’. For a guy who we are meant to believe has a solid grasp of logic, reason, argument, and science, this seems a rather appalling justification for his beliefs. But of course, nobody is perfect, and we need to judge each argument on its merits.