THE TEMPTATION OF QUANTIFICATION
Last week the L.A. Times rated Los Angeles school teachers by their pupils’ scores on two standardized tests. On Friday, the NY Times discussed the abysmal track record of Wall Street’s quantitative analysts, known as “quants.” What do the two stories have in common? Our compulsion to measure the unmeasurable.
The legendary management and statistics guru W. Edwards Deming is often quoted as saying “You can’t manage what you can’t measure.” That quote has even become part of the curriculum at Wharton School of Business, where it is taught as a cornerstone of business analysis.*
The problem is that Prof. Deming never said that. In fact, what he said was exactly the OPPOSITE — “the most important figures that one needs for management are unknown or unknowable.”*
Which brings us back to teachers and Wall Street wizards. It is very tempting to believe that simple mathematical models can yield answers to complex questions. It’s even more tempting to believe that complex mathematical models can yield answers to complex questions.
But any model is inherently flawed, for two specific reasons —
1) No model of a complex system is anywhere near as complex as the real world (and the “quants” models of the markets were so complex they required more computing power than NASA)
2) Any model, simple or complex, rests on basic assumptions about what variables should be measured. The fewer the variables (say, the scores on two standardized tests), the less accurate the measurement.
Even with insanely complicated models like those used by the “quants,” those fundamental assumptions can make the results worthless. Among other things, the quants neglected to model the fact that in an unregulated and unobserved market, some players are going to cheat. And cheat big time.
Oooops. There goes $700 billion in investor value.*** And the world economy is still reeling.
An analogy could be helpful here, as they often are. Take two cars – the Jaguar XKR 175, and the Subaru Impreza WRX sti. Take the three statistical measurements that are commonly used to assess the value of a car – 0-60 acceleration, miles per gallon, and price. Compare and contrast.
Jaguar – 4.6 seconds in the speed test, 16 – 20 mpg, city and highway, price $104,500
Subaru – 4.4 seconds, 17-23 mpg, $34,555.
Clearly, the Subaru is a superior automobile. I have proven it statistically. So you know all those Wall Street quants (and the statistician hired by the LA Times) drive Subarus. Some of the teachers do, too.
*Knowledge@Wharton, Sept. 6.2006 )http://knowledge.wharton.upenn.edu/article.cfm?articleid=1546)
**Out of the Crisis, W. Edwards Deming, p. 121
***New York Times, 8/20/10, p. b1
Ted Bellamy is the nom-de-plume of Scott Wyant, who’s lived in Culver City for more than 20 years. Not only does he know what he’s about, he knows how to use footnotes.
Be the first to comment