The Infamous graph and the Law of Triviality
The 2008 prediction of the effect of the ARRA on the unemploymemt rate was wildly off. Why?
Contrary to what one might expect, they did not model the effect of the stimulus on the unemployment rate at all. Instead they modeled the effect on GDP:
...the final step was to take the effect on GDP and translate it into job creation. We use the relatively conservative rule of thumb that a 1% increase in GDP corresponds to an increase in employment of 0.75%. This has been the rough correspondence over history and matches the FRB/US model reasonably well. http://otrans.3cdn.net/45593e8ecbd339d074_l3m6bt1te.pdf
The basis of the estimate was the empirical Okun's law, http://en.wikipedia.org/wiki/Okun's_law
This "law" worked well between 1948 and 1995. However, over the last three recessions it did not work
http://www.kansascityfed.org/Publicat/ECONREV/PDF/4q07Knotek.pdf
with the coefficients deviating from the historical average. Looking at the Okun's plot past year 2000
http://delong.typepad.com/sdj/2010/02/the-life-and-strange-death-of-okuns-law.html
http://www.j-bradford-delong.net/movable_type/2003_archives/002121.html
it is evident that (for whatever reason) this law broke down and can no longer serve as the basis for any predictions. Delong stressed that back in 2003, and yet... his own UCB colleagues used this law, in full knowledge that it does not work. Had they correctly predicted GDP growth, the predictions of job losses would still be grossly inaccurate.
However, they also failed to project the increase in GDP. It was estimated using multipliers for direct and indirect effects of federal spending (how increased federal spending in a given industry results in wealth growth). It is stunning how primitive was the approach:
...we started by averaging the multipliers for increases in government spending and tax cuts from a leading private forecasting firm and the FRB/US model. The two sets of multipliers are similar and are broadly in line with other estimates. We considered multipliers for the case where the federal funds rate remains constant, rather than the usual case where the Federal Reserve raises the funds rate in response to fiscal expansion, on the grounds that the funds rate is likely to be at or near its lower bound of zero for the foreseeable future. (Appendix 1)
They've polled forecasting firms and averaged expert's estimates. These averages were "more or less" in line with the FRB/US model. The assumption of the flat rate is curious, because (just as they admit) the Fed does not spend in this way. As such estimates still did not look politically acceptable, further adjustments were made to estimate the effect of transfer of the federal funds to the states. Here no expert opinion even existed, so the effect was assumed to be such and such, without any rationale at all. The authors themselves admitted that it was garbage in garbage out:
...We applied these multipliers directly to the straightforward elements of the package, but made some adjustments for elements that take the form of transfers to the states and tax-based investment incentives. For transfers to the states, we assumed that 60% is used to prevent spending reductions, 30% is used to avoid tax increases, and the remainder is used to reduce the amount that states dip into rainy day funds. We assumed that these effects occur with a one quarter lag. For tax-based investment incentives, we used the rule of thumb that the output effects correspond to one-fourth of the effects of an increase in government spending with the same immediate revenue effects. This implies a fairly small effect from a given short-term revenue cost of the incentives. But, because much of the lost revenue is recovered in the long run, it implies a fairly substantial short-run impact for a given long-run revenue loss. We confess to considerable uncertainty about our choice of multipliers for this element of the package.
Just think about it... The US economy was in peril. The President appointed the leading academic economists to advise him on economic policy. What did they do? They did an hour worth of spreadsheet manipulation.
The apologists of technocracy never tire of stressing how important it is that the most informed people make the most momentous decisions. Here is a glimpse how such decisions are being made. It is two novices to fiscal governance doing corner-of-the-envelope calculations using meaningless numbers; their sterling academic credentials used to persuade laymen that their numerology is "science". The higher level decision is being made, the lower amount of expertise goes into it. The tightest scrutiny is reserved for the least important, mundane decisions.
I see it constantly. A $100k grant application requires 50-100 pages of terse documentation, several rounds of review, panels of experts arguing about the responsible way of spending peanuts. By contrast, $10M federal contract requires a single-page white paper; the decision is made by very few people. A person who decides on $10M does not have time to read 100 pages of documentation and study the contradicting expert opinion.
It is the Parkinson's Law of Triviality in action.
http://en.wikipedia.org/wiki/Parkinson's_Law_of_Triviality