Computer models and cognitive failure
One of the more amusing aspects of the current credit crisis is the massive failure of relying on computer models for assessing risk.
Financial institutions used highly sophisticated computer models, put together by highly-paid people using masses of data based on what was taken to the most up-to-date understanding of how things work. All of which gave the output of the models huge credibility.
The problem was precisely that they had such credibility. In particular, their output was treated as empirical evidence: as telling people about the state of their risk exposure.
They did, of course, nothing of the kind. All they did-all computer models can ever do-is tell you the consequences of your premises, both empirical and analytical/causal. They do not tell you about how the world is. They tell you about how you think the world is. One can then test your thinking about the world by comparing what your model(s) churn out to how the world turns out to be.
But this is a distinction that is very, very easy to lose sight of. Particularly given the effort and analytical power put into creating the things and their “black box”-facts in one end, results out the other end-nature.
Any analytical discipline that comes to rely on computer models is in deep, deep trouble: particularly if they are treated as providing empirical evidence. And the more there are things that matter in understanding some system that are poorly understood, the deeper the trouble.
In the case of the financial institutions, clearly the various financial instruments being modelled were not understood anywhere near as well as people thought. The modellers covered them, they just did not cover them accurately. So all the models ended up showing was (after fact) what their modellers did not understand: their “unknown unknowns” (i.e. they apparently did not know that they did not know).
The case is hardly better if one has known unknowns: things that you know that you do not know. Since one then has to put “patches” in one’s computer models to “guess” at their operation. Guesses that are unlikely to be accurate but which the existence of the computer model disguises them as being guesses.
Such as, for example: the oceans and their circulations are the thermal and inertial flywheels of the climate system; as the ocean circulation changes, the atmosphere and its climate respond. Our knowledge of subsurface ocean circulations and their variability is limited. Without this vital input, projections of future climate are tenuous at best.
Our climate models are not scientific predictions, still less projections. They are guesses. Partially informed, fairly sophisticated, guesses: but guesses. Guesses that them being computer models by clever people disguise being guesses. Exactly how many billions of dollars are such guesses worth the wager of? There may be a few folk on Wall St who can give you a heads up on that.
Financial institutions used highly sophisticated computer models, put together by highly-paid people using masses of data based on what was taken to the most up-to-date understanding of how things work. All of which gave the output of the models huge credibility.
The problem was precisely that they had such credibility. In particular, their output was treated as empirical evidence: as telling people about the state of their risk exposure.
They did, of course, nothing of the kind. All they did-all computer models can ever do-is tell you the consequences of your premises, both empirical and analytical/causal. They do not tell you about how the world is. They tell you about how you think the world is. One can then test your thinking about the world by comparing what your model(s) churn out to how the world turns out to be.
But this is a distinction that is very, very easy to lose sight of. Particularly given the effort and analytical power put into creating the things and their “black box”-facts in one end, results out the other end-nature.
Any analytical discipline that comes to rely on computer models is in deep, deep trouble: particularly if they are treated as providing empirical evidence. And the more there are things that matter in understanding some system that are poorly understood, the deeper the trouble.
In the case of the financial institutions, clearly the various financial instruments being modelled were not understood anywhere near as well as people thought. The modellers covered them, they just did not cover them accurately. So all the models ended up showing was (after fact) what their modellers did not understand: their “unknown unknowns” (i.e. they apparently did not know that they did not know).
The case is hardly better if one has known unknowns: things that you know that you do not know. Since one then has to put “patches” in one’s computer models to “guess” at their operation. Guesses that are unlikely to be accurate but which the existence of the computer model disguises them as being guesses.
Such as, for example: the oceans and their circulations are the thermal and inertial flywheels of the climate system; as the ocean circulation changes, the atmosphere and its climate respond. Our knowledge of subsurface ocean circulations and their variability is limited. Without this vital input, projections of future climate are tenuous at best.
Our climate models are not scientific predictions, still less projections. They are guesses. Partially informed, fairly sophisticated, guesses: but guesses. Guesses that them being computer models by clever people disguise being guesses. Exactly how many billions of dollars are such guesses worth the wager of? There may be a few folk on Wall St who can give you a heads up on that.