the NUMERATI, by Stephen Baker
Subtitle: there isn't one. Whoa! What's up with that, Stephen Baker? All non-fiction books have a subtitle, this century. It's like a law or something. Also, the lower case "the" and upper case "NUMERATI". This man is a language arts radical.
First off, this book was published in 2008, which given the topic makes it somewhat archaic. This made it a more interesting read, in many ways, and also gave me some insight into how well he was interpreting the trends he was writing about.
For example, when he writes about the Obama campaign (v1), he was not able to say for sure if it would be a successful one. Hindsight tells us that it was, sure enough, an advance over previous (and competing) presidential campaigns in terms of how it utilized data on potential voters.
On the other hand, when he writes about the company that was working on making a talking shopping cart, which could make suggestions or offer coupons in real time, based on our purchase history, we have the knowledge that this did not, in fact, change the face of grocery shopping.
In case the title doesn't make it clear, this book is about, well, people sort of like me. People who can program, who love crunching data. In particular, though, he's interested in the idea that a class of such people are going to exert more and more control over society. He begins with examples of some of the more invasive and manipulative uses: how to use Big Data to decide which workers are more valuable than others, to decide which voters are most likely to be influenced by the right pitch, and how to best manipulate us into buying things we otherwise wouldn't.
But, then, he moves into fields where Big Data, and analysis of ever-increasing sophistication (in 2008 this level wasn't very high), can work for us. For example, helping us spot medical problems in ourselves much earlier, or helping us find our soul match online. These latter two objectives, while less creepy and manipulative than political or marketing campaigns that know how to manipulate our emotions better, are also still (in 2013) largely aspirational. But then, the percentage of new marriages which result from online pairings continues to go up, so who knows.
Reading this book was, for me, kind of like reading a foreigner's account of a trip in the U.S.A. Baker is not a programmer, and probably not really much of a numbers guy. He is a journalist, and his viewpoint on the world of programmers and data crunchers was decidly an outsider's one. That doesn't mean it wasn't informed, or informative; in some ways it was more informative. Just don't expect to learn much about the details of the algorithms used, or the math behind them.
He could ask a good question, though. When interviewing James Schatz, the NSA's chief mathematician (is that a title?), he tells him about a p oint raised by Prabhakar Raghavan, chief of research at Yahoo, that there is an optimum amount of data, and an optimum amount of detail in that data, to get the right information out of it.
The response of Schatz, of the NSA, is "More data is always better". He even seems taken aback by the question. If that's really what the NSA's chief mathematician thinks, then the NSA is considerably less competent than I thought. "More data is always better"? I'm not a chief mathematician anywhere, and I know that's not true. So do pollsters; information on the opinions of the right 5000 people is a lot better than an internet poll on CNN from 5,000,000 people. More data is not always better. But, unfortunately, the best of the Numerati are not calling the shots at the NSA, it appears.
Which really is the issue with the Numerati. It's not that they can look at lots of data and discover things we didn't know before. It's that they could think they have, or their bosses could think they have, and it might be wrong. Genetic testing of evidence from crime scenes is good, unless the people conducting the tests get it wrong, and the prosecutor and jury are convinced that genetic testing is wrong one time in a million. Similarly, if the Numerati attempting to find terrorists before they streak tell the NSA, CIA, or FBI that someone is worth looking into, will they assume that the Computer must be right?
Baker's book is a reasonable introduction to the idea of how access to more and more data, and more and more horsepower to analyze that data, will impact many different parts of our lives. Even the parts (like talking shopping carts) which look off now, are informative about what sorts of potential uses are NOT likely to take off (personalized talking billboards a la "Minority Report" are probably not either). There's more (and more in-depth) analysis needed here, but Baker gets beyond the initial gee-whiz or oh-no stories, and that's a step in getting our collective heads around the world into which we're headed.
First off, this book was published in 2008, which given the topic makes it somewhat archaic. This made it a more interesting read, in many ways, and also gave me some insight into how well he was interpreting the trends he was writing about.
For example, when he writes about the Obama campaign (v1), he was not able to say for sure if it would be a successful one. Hindsight tells us that it was, sure enough, an advance over previous (and competing) presidential campaigns in terms of how it utilized data on potential voters.
On the other hand, when he writes about the company that was working on making a talking shopping cart, which could make suggestions or offer coupons in real time, based on our purchase history, we have the knowledge that this did not, in fact, change the face of grocery shopping.
In case the title doesn't make it clear, this book is about, well, people sort of like me. People who can program, who love crunching data. In particular, though, he's interested in the idea that a class of such people are going to exert more and more control over society. He begins with examples of some of the more invasive and manipulative uses: how to use Big Data to decide which workers are more valuable than others, to decide which voters are most likely to be influenced by the right pitch, and how to best manipulate us into buying things we otherwise wouldn't.
But, then, he moves into fields where Big Data, and analysis of ever-increasing sophistication (in 2008 this level wasn't very high), can work for us. For example, helping us spot medical problems in ourselves much earlier, or helping us find our soul match online. These latter two objectives, while less creepy and manipulative than political or marketing campaigns that know how to manipulate our emotions better, are also still (in 2013) largely aspirational. But then, the percentage of new marriages which result from online pairings continues to go up, so who knows.
Reading this book was, for me, kind of like reading a foreigner's account of a trip in the U.S.A. Baker is not a programmer, and probably not really much of a numbers guy. He is a journalist, and his viewpoint on the world of programmers and data crunchers was decidly an outsider's one. That doesn't mean it wasn't informed, or informative; in some ways it was more informative. Just don't expect to learn much about the details of the algorithms used, or the math behind them.
He could ask a good question, though. When interviewing James Schatz, the NSA's chief mathematician (is that a title?), he tells him about a p oint raised by Prabhakar Raghavan, chief of research at Yahoo, that there is an optimum amount of data, and an optimum amount of detail in that data, to get the right information out of it.
The response of Schatz, of the NSA, is "More data is always better". He even seems taken aback by the question. If that's really what the NSA's chief mathematician thinks, then the NSA is considerably less competent than I thought. "More data is always better"? I'm not a chief mathematician anywhere, and I know that's not true. So do pollsters; information on the opinions of the right 5000 people is a lot better than an internet poll on CNN from 5,000,000 people. More data is not always better. But, unfortunately, the best of the Numerati are not calling the shots at the NSA, it appears.
Which really is the issue with the Numerati. It's not that they can look at lots of data and discover things we didn't know before. It's that they could think they have, or their bosses could think they have, and it might be wrong. Genetic testing of evidence from crime scenes is good, unless the people conducting the tests get it wrong, and the prosecutor and jury are convinced that genetic testing is wrong one time in a million. Similarly, if the Numerati attempting to find terrorists before they streak tell the NSA, CIA, or FBI that someone is worth looking into, will they assume that the Computer must be right?
Baker's book is a reasonable introduction to the idea of how access to more and more data, and more and more horsepower to analyze that data, will impact many different parts of our lives. Even the parts (like talking shopping carts) which look off now, are informative about what sorts of potential uses are NOT likely to take off (personalized talking billboards a la "Minority Report" are probably not either). There's more (and more in-depth) analysis needed here, but Baker gets beyond the initial gee-whiz or oh-no stories, and that's a step in getting our collective heads around the world into which we're headed.