Playoff talk
Sigh. Do we have to continue to hear about how, based on the fact that it is possible for a high-payroll team to lose a playoff series against a low-payroll team, then payroll is supposedly more or less a nonfactor in Major League Baseball? This is absurd. This is the DEFINITION of a small sample size. Bill James liked to ponit out that over a two week span, there is a 40% chance that a .275 hitter will outhit a .300 hitter. In other words, you cannot judge the quality of a team by just a few games, including in the postseason.
But people love to take the irrational stance that whoever won must have been better, not really realizing that this is like saying heads was more likely to come up on the coin I just flipped, since that was the outcome. Read a book, sportswriters!
I know just the book, too. I found something at the bookstore the other day, and though I didn't buy it (bank account is dangerously close to zero), I read most of it in the store. The book: Supercrunchers: Why Thinking-by-numbers is the New Way to Be Smart by Ian Ayres. Essentially, this book is a big slap in the face to intuitionists. These days, in just about any field, from medicine to law to bookselling, it is possible for a sophisticated enough algorithm to make more accurate decisions than a team of experts. We all already knew this was true of the stock market and baseball (well, the "baseball experts" tend not to know...), but I was a bit surprised at some of the other instances.
For instance, a team of people wrote a program that out-performed a cross section of legal scholars, most of whom held chaired positions at law schools or were clerks for Supereme Court Justices, at predicting the outcomes of cases tried by the Supreme Court. Not only did the program win handily at overall decisions, it also picked the correct vote of individual members of the Court, including those of key swing votes. (Sandra Day O'Connor was the case study talked about by the author. On a side note, I miss her.)
A medical diagnostic program called "Isabel" also demonstrated that, given enough rigorous input, it could take a set of symptoms and come up with a short list that includes the correct diagnosis 96% of the time. Obviously, this isn't going to make doctors obsolete, but it will make their jobs easier, and it will help people survive.
Other cases tackled in the book include mostly correct predictions of how many lives could be (and were) saved by the Evidence-Based Medicine movement and how individuals can do simple calculations using things like Bayes's theorem to outsmart intuition. Why does this work? Two reasons: mathematics and objective reality.
Mathematics is a tool for looking at reality in creative ways without letting extraneous (and usually false) assumptions color our outlook. Not only do we have biases that we learn, we as humans have innate tendancies to have poor statistical intuition. (And we can demonstrate this by analyzing psychology experiments, such as one included in the book. Given a list of ten quantities that most people won't know off the top of their heads, readers are asked to estimate 90% confidence intervals for potential values of these quantities. Not surprisingly, most people end up with between four and seven of the ten somewhere outside of the confidence interval, which shows that people are overconfidenct in their estimations.)
Objective analysis of situations essentially allows us to say "what is likely to happen here?" rather than "I'm guessing this will happen." It allows us to take into account prior distributions, contrary to what a nurse, for example, might say to a poor patient getting a positive result on a cancer test with 80% accuracy. (People tend to assume that means there is a 70%-80% chance they have cancer, when in reality about 99% of people who get tested do NOT have cancer, and taking that and the false positive rate into account, a positive test only means you are about 7%-10% likely to have cancer, depending on a couple of variables. MOST people get this wrong, but mathematics clearly indicates the right answer.)
Overall, the book is good as a popular science book intended to introduce people to statistics with motivation to pursue the details, though it would be extremely unwise to say that you'll be an expert in anything after finishing the book. Rather, you should have your appetite whetted for something with a bit more meat to it, like an introductory statistics textbook. Those who approach the book with some background in math or statistics probably won't learn all that much, but it's still fun to read about applications.
So, returning to my original point, a baseball analyst might watch the playoffs and conclude that the Indians are a better team than the Yankees, based on the fact that they looked better in the extremely small sample taken. This is baloney. I repeat: this is utter BALONEY. While it's certainly *possible* that the conclusion is true, the path taken to get to the conclusion is so ridiculously misguided that it's really difficult to trust the conclusion at all. When I look at the Yankees, I see a powerhouse team that has bought all the offense it needs to go to the playoffs EVERY YEAR, while the Indians are the latest flash in the pan from the AL Central division. It is impossible to say, based on only these few observations and ignoring the strong evidence to the contrary (the last few seasons), that the Indians are even worthy of walking onto the same field as the Yankees.
No, all we can say is that the Indians had a nice playoff series. Even crummy players can do better than their superiors on some occasions.
And if none of this convinces you, just remember that last year, David Eckstein won the World Series MVP award. 'Nuff said, there.
But people love to take the irrational stance that whoever won must have been better, not really realizing that this is like saying heads was more likely to come up on the coin I just flipped, since that was the outcome. Read a book, sportswriters!
I know just the book, too. I found something at the bookstore the other day, and though I didn't buy it (bank account is dangerously close to zero), I read most of it in the store. The book: Supercrunchers: Why Thinking-by-numbers is the New Way to Be Smart by Ian Ayres. Essentially, this book is a big slap in the face to intuitionists. These days, in just about any field, from medicine to law to bookselling, it is possible for a sophisticated enough algorithm to make more accurate decisions than a team of experts. We all already knew this was true of the stock market and baseball (well, the "baseball experts" tend not to know...), but I was a bit surprised at some of the other instances.
For instance, a team of people wrote a program that out-performed a cross section of legal scholars, most of whom held chaired positions at law schools or were clerks for Supereme Court Justices, at predicting the outcomes of cases tried by the Supreme Court. Not only did the program win handily at overall decisions, it also picked the correct vote of individual members of the Court, including those of key swing votes. (Sandra Day O'Connor was the case study talked about by the author. On a side note, I miss her.)
A medical diagnostic program called "Isabel" also demonstrated that, given enough rigorous input, it could take a set of symptoms and come up with a short list that includes the correct diagnosis 96% of the time. Obviously, this isn't going to make doctors obsolete, but it will make their jobs easier, and it will help people survive.
Other cases tackled in the book include mostly correct predictions of how many lives could be (and were) saved by the Evidence-Based Medicine movement and how individuals can do simple calculations using things like Bayes's theorem to outsmart intuition. Why does this work? Two reasons: mathematics and objective reality.
Mathematics is a tool for looking at reality in creative ways without letting extraneous (and usually false) assumptions color our outlook. Not only do we have biases that we learn, we as humans have innate tendancies to have poor statistical intuition. (And we can demonstrate this by analyzing psychology experiments, such as one included in the book. Given a list of ten quantities that most people won't know off the top of their heads, readers are asked to estimate 90% confidence intervals for potential values of these quantities. Not surprisingly, most people end up with between four and seven of the ten somewhere outside of the confidence interval, which shows that people are overconfidenct in their estimations.)
Objective analysis of situations essentially allows us to say "what is likely to happen here?" rather than "I'm guessing this will happen." It allows us to take into account prior distributions, contrary to what a nurse, for example, might say to a poor patient getting a positive result on a cancer test with 80% accuracy. (People tend to assume that means there is a 70%-80% chance they have cancer, when in reality about 99% of people who get tested do NOT have cancer, and taking that and the false positive rate into account, a positive test only means you are about 7%-10% likely to have cancer, depending on a couple of variables. MOST people get this wrong, but mathematics clearly indicates the right answer.)
Overall, the book is good as a popular science book intended to introduce people to statistics with motivation to pursue the details, though it would be extremely unwise to say that you'll be an expert in anything after finishing the book. Rather, you should have your appetite whetted for something with a bit more meat to it, like an introductory statistics textbook. Those who approach the book with some background in math or statistics probably won't learn all that much, but it's still fun to read about applications.
So, returning to my original point, a baseball analyst might watch the playoffs and conclude that the Indians are a better team than the Yankees, based on the fact that they looked better in the extremely small sample taken. This is baloney. I repeat: this is utter BALONEY. While it's certainly *possible* that the conclusion is true, the path taken to get to the conclusion is so ridiculously misguided that it's really difficult to trust the conclusion at all. When I look at the Yankees, I see a powerhouse team that has bought all the offense it needs to go to the playoffs EVERY YEAR, while the Indians are the latest flash in the pan from the AL Central division. It is impossible to say, based on only these few observations and ignoring the strong evidence to the contrary (the last few seasons), that the Indians are even worthy of walking onto the same field as the Yankees.
No, all we can say is that the Indians had a nice playoff series. Even crummy players can do better than their superiors on some occasions.
And if none of this convinces you, just remember that last year, David Eckstein won the World Series MVP award. 'Nuff said, there.