Confirmation bias
There's an interesting example of confirmation bias in the back of the July Scientific American, which is amusing because it comes just two pages after Michael Shermer's column in which he talks about confirmation bias and why skepticism is important to science. The article asserts that child mortality rates decline as women become better educated, and asserts that this is becauseeducated women "make wiser choices about hygiene, nutrition, immunization and contraception".
Here's the online version of the article. Notice anything about it?
Look carefully. Pay particular attention to Niger, Paraguay, Fiji, Namibia, Tonga, the Marshall Islands, New Zealand, Ukraine, the Phillipines, Burkina Faso, Ethiopia, Chad, Saint Lucia, Equatorial Guinea. Compare to, say, the Maldives, Portugal, Nepal, Montenegro. Once I've pointed these out, I'm sure you should be able to find other similar examples. There's a lot of them.
So what do these examples I've pointed out show?
Well, it's more what they DON'T show. Which is to say, there is no clear correlation between increases in education of women and decreases in child mortality. Some of these have significant increases in length of education with virtually no change in child mortality. Some have almost no change in education and significant decreases in child mortality. Some show little change in either. Some countries with almost no improvement in education show larger decreases in child mortality than other countries in which female education increased much more.
Gambia, for example, started out in 1970 with a higher child mortality rate than nearby Ghana, yet improved its child mortality rate by 2009 twice as much as Ghana did, even though Ghana had around twice the improvement in education. During the same period, Nigeria and the Marshall Islands both improved their education just as much as Ghana did, with no change whatsoever in child mortality. Somalia and Rwanda both had significant improvements in child mortality, while Equatorial Guinea, which improved education twice as much as either, showed no child mortality reduction at all. Chad improved childhood mortality more than Equatorial Guinea did, despite almost no improvement in education and an ongoing civil war, while Burkina Faso showed more child mortality reduction with even less improvement in education. Niger matched Burkina Faso's gains despite no improvement in education at all. Turkmenistan and neighboring Uzbekistan show almost identical improvements in education, but Turkmenistan shows an almost 3:1 drop in child mortality while Uzbekistan's hardly changed. Child mortality plunged in Afghanistan while education of women remained static. Fiji made smaller gains in child mortality than Australia, despite larger gains in education. Samoa and nearby Tonga improved their education by almost the same amount, but Samoa's child mortality rate declined, while Tonga's did not. Both are eclipsed by Vanuatu, which improved education less than either. Portugal improved child mortality twice as much as next-door Spain, on almost identical improvements in education. Guyana vs. Haiti, Chile vs. Uruguay, Saudi Arabia vs. Lebanon. Bosnia-Herzegovina and Montenegro, despite two Balkan wars, left the much wealthier Russian Federation and Ukraine in their dust. War-torn El Salvador passed Mexico by, while Panama scarcely improved.
The authors of this study desperately want to find a causative link between education of women and reduced child mortality. However, their published results do not support their conclusion. There exists a correlation, yes - albeit a weak one - but correlation does not imply causation. In fact, what all of these results have in common is that decrease in child mortality is more strongly correlated to time than it is to improvements in education of women.
Time alone, of course, doesn't do anything, and cannot be responsible. But it makes a nice control. There is another factor at work here, probably several other factors; but the authors do not appear to have looked for them, because they want their explanation to be right. But when your results appear to show that your data is actually more weakly correlated to your proposed cause than it is to time alone, you have a credibility problem.
This entry was originally posted at http://unixronin.dreamwidth.org/814822.html. That post currently has
comments.
You may comment there via OpenID even if you do not have a Dreamwidth account.