Skepticism for fun and profit.
I've been somewhat alarmed by how skepticism has got a bad rap these days. The Gradstudents who have their evolution topics course would be the first thing that springs to mind - yesterday, I pressed the group of them to start asking how they'd evaluate claims made by West-Eberhard, and whether her predictions were strong enough to warrant being called such. They sorta shrugged off the efforts, and made vague mutterings to the effect of that they were accepting it as-is. What an astonishing thing for scientists to say: god knows lots of garbage gets published. Are they as uncritical about that as well?
I want to use two examples from people who read this blog - I'd ask both of you don't take offence, but I know it's impossible to ask someone not to be angry. Just realize that my skepticism and incredulity isn't directed at you, but at the people who fed you suspect information.
Let's start with a claim from a church group that wants to feed the poor - an admirable goal. The first claim was that they could feed a child for 6 months for just 30 dollars - maybe, or perhaps not, but I'll be passing this one up. The next claim was that they could provide all the needed food for 10,000 adults in Zimbabwe alone. This is when my sonar started returning pings.
10,000 adults. That's a lot of calories. That's a lot of food. Just how much food? Let's say they get 300 grams of rice a day per adult. That's seems like a lot, until you consider low-grade rice only has (give or take) 353 Calories per 100 grams, for a total of 1059 Calories. That seems like a reasonable number for a food agency to shoot for, energy-wise.
So that gives us 300 grams/day * 365 days/year * 10000. For a total of 1,095,000,000 gram/year of rice. 1.095 million kilograms of rice is no number to sneeze at. Now, looking up the price of rice, I see a lot of numbers being flung around. If I were to take the mode, though, I'd say 800 USD per metric ton. That's a total cost of 876k USD/year for the food cost alone. Now, when it comes to shipping large quantities, I'm even more clueless about it than I am commodities, but let's say, for the sake of argument, it costs 80 USD/ton to get from where the rice is produced to Zimbabwe. That's 87.6k USD/year, for a total of 963.6k USD/year. For just buying and transporting the rice required.
Is this one church really raising a million dollars a year (nearly 20k/week in donations?), in addition to all their other charitable projects (because they allegedly feed people in the US, too, and presumably cover their own operating expenses)? And then I'm told that the food they supply a number of other things in it, like process soy, flavouring, etc. I'm just imagining the price spiralling up at a precipitous rate, especially when you include fuel adjustments, logistic prices, the real estate to operate out of, and god forbid the manpower actually required run this operation for a year. Even assuming you don't need to pay your staff, I'm betting they, themselves, will want to eat at a bare minimum.
This whole thing becomes quickly suspect. What's more likely is that they're feeding 10,000 people under a very different assumption of the meaning 'feeding;' that they're making a load of assumptions in their calculations, which is how they reach such a staggering number with one little church. Whatever the explanation, it's dubious that this task is actually being accomplished by a single, normal church full of every-day people.Next up is another food related one. Wine. A while ago, there was a thing circulating in various ethanol lovers groups that pesticides were making their way into wine. I made like a salmon and swum upstream, keeping the scent of the head waters in my nostrils until I found the original study (How /do/ salmon smell, anyhow? I never really looked that closely, but I'm pretty sure they don't have nares). I found the study and implore any analytical chemists not to read it, as their eyes will bleed.
• The 'study' was performed by the Pesticide Action Network. Bias immediately becomes an issue. In a /real/ study, the 'conflicting interest' section here would be as long as the actual study.
• It was released as the form of a press release. *cough*
• No indication of what authors oversaw the analysis. Just 'commercial laboratories.'
• Not even a mention of how they did it. HPLC chromatography with UV-Vis? LC/Mass spec work? Antibody assays? And their criteria for buying cheap wine invites low quality to begin with - not nearly enough was said about how they picked their samples. How you did it really matters a lot in interpreting results.
Using a bit of an exaggeration to demonstrate my point, if I say I collected 3 fish on my way from BC to Washington, you might conclude fish stocks collapsed. However, it also could have been that I drove from BC to WA, and I used a nalgene and a ham-sandwich to trap wherever I crossed a river. Methods matter.
• Now we're onto results. And the biggest, most mind blowing thing is... there's absolutely no discussion of the uncertainties involved. BLUH! The hell?!
If methods matter, uncertainties are astoundingly critical in these manners of analyses. 34 of 40 bottles contained pesticide residues. Really? How did you determine that? What constitutes 'presence?' To a non-analytical chemist, these questions might seem banal, but it /really matters/. There are no perfect methods that tell you exactly how much was contained within. Even your sampling screws you, and you need to make a statement about how much variability exists.
Example. Okay, let's say they detected 1 µg/L of compound X. What is the uncertainty on that measurement? If it's ±0.1 µg/L, you can probably conclude that there's actually some there, and the quantity is accurate. However, if the 95% confidence interval is ±2µg/L on the measurement... can you really say you have any at all? It's not distinguishable from zero when the error is around comparable to the measurement.
And this is when they give measurements at all! In some places, it's just 'was detected. My favourite was Pessac-Léognan Cru Classé, which contains 2µg/L pyrimethanil, and 5.8 µg/L procymidone, but (trace) fenhexamide. Well what the bloody hell does (trace) mean? Are we talking ng/L? pg/L? We saw a blip on the chromatogram, and we're calling it fenhexamide? Ugh!
They give no magnitude on most, just a discussion of 'it's an eeeeendocrine disruptor! Spooooooky!' Yes. It is. In clinical doses. But since they don't tell you what 'detected' means, never-mind actual amounts, you don't know if it comes anywhere close to significant doses.
Here's a panic inspiring fact: Your drinking water contains mercury. Quick, run for the hills! We're all going to die! No, not really, because the drinking water (hopefully) contains so little mercury that 'eh', it /really/ doesn't matter. I mean really, really doesn't matter.
Now, with the 'world famous wines', they actually give numbers, and if you're so motivated, you could hunt down what doses are required for effects. I'm not going to, because I have no way of knowing if those numbers are even imaginably accurate. That table is so half-assed that I couldn't help but notice that in Pessac-Léognan Cru Classé, pyrimethanil has only one sig-fig, whereas other mentions of it have at least two, or more (in fact, everything else on the table except (trace) has one trailing decimal).
You know what I'm going to do? I'm going to have a drink. I'm so underwhelemed by this black-box 'study' by a biased group that I'm resigning it to the junk-pile of studies, and taking my chances with some pesticide-laden ethanol. How about this one, instead? It's a review of numerous well vetted, peer-reviewed studies that suggest that wine drinking is beneficial to the health, in moderation. The mechanism, to my knowledge, remains unclear. Apparently these people didn't get the memo that the hundreds of pesticides are supposed to be harming their subjects.
In both cases, a group is making a claim, and one people might be inclined to accept. However, if you start poking at it, you realize that with the evidence provided, the story doesn't hold weight; just a little logical inference (and a pad of paper to do a dimensional analysis) makes the whole thing seem implausible.
But there seems to be some impression that skeptics are party poopers and cynics. Or worse still, know-it-all jerks. It's not. Skepticism insulates you from bad information, which can be more damaging that a lack of information. I only wish more people would but information through the grinder.
I want to use two examples from people who read this blog - I'd ask both of you don't take offence, but I know it's impossible to ask someone not to be angry. Just realize that my skepticism and incredulity isn't directed at you, but at the people who fed you suspect information.
Let's start with a claim from a church group that wants to feed the poor - an admirable goal. The first claim was that they could feed a child for 6 months for just 30 dollars - maybe, or perhaps not, but I'll be passing this one up. The next claim was that they could provide all the needed food for 10,000 adults in Zimbabwe alone. This is when my sonar started returning pings.
10,000 adults. That's a lot of calories. That's a lot of food. Just how much food? Let's say they get 300 grams of rice a day per adult. That's seems like a lot, until you consider low-grade rice only has (give or take) 353 Calories per 100 grams, for a total of 1059 Calories. That seems like a reasonable number for a food agency to shoot for, energy-wise.
So that gives us 300 grams/day * 365 days/year * 10000. For a total of 1,095,000,000 gram/year of rice. 1.095 million kilograms of rice is no number to sneeze at. Now, looking up the price of rice, I see a lot of numbers being flung around. If I were to take the mode, though, I'd say 800 USD per metric ton. That's a total cost of 876k USD/year for the food cost alone. Now, when it comes to shipping large quantities, I'm even more clueless about it than I am commodities, but let's say, for the sake of argument, it costs 80 USD/ton to get from where the rice is produced to Zimbabwe. That's 87.6k USD/year, for a total of 963.6k USD/year. For just buying and transporting the rice required.
Is this one church really raising a million dollars a year (nearly 20k/week in donations?), in addition to all their other charitable projects (because they allegedly feed people in the US, too, and presumably cover their own operating expenses)? And then I'm told that the food they supply a number of other things in it, like process soy, flavouring, etc. I'm just imagining the price spiralling up at a precipitous rate, especially when you include fuel adjustments, logistic prices, the real estate to operate out of, and god forbid the manpower actually required run this operation for a year. Even assuming you don't need to pay your staff, I'm betting they, themselves, will want to eat at a bare minimum.
This whole thing becomes quickly suspect. What's more likely is that they're feeding 10,000 people under a very different assumption of the meaning 'feeding;' that they're making a load of assumptions in their calculations, which is how they reach such a staggering number with one little church. Whatever the explanation, it's dubious that this task is actually being accomplished by a single, normal church full of every-day people.Next up is another food related one. Wine. A while ago, there was a thing circulating in various ethanol lovers groups that pesticides were making their way into wine. I made like a salmon and swum upstream, keeping the scent of the head waters in my nostrils until I found the original study (How /do/ salmon smell, anyhow? I never really looked that closely, but I'm pretty sure they don't have nares). I found the study and implore any analytical chemists not to read it, as their eyes will bleed.
• The 'study' was performed by the Pesticide Action Network. Bias immediately becomes an issue. In a /real/ study, the 'conflicting interest' section here would be as long as the actual study.
• It was released as the form of a press release. *cough*
• No indication of what authors oversaw the analysis. Just 'commercial laboratories.'
• Not even a mention of how they did it. HPLC chromatography with UV-Vis? LC/Mass spec work? Antibody assays? And their criteria for buying cheap wine invites low quality to begin with - not nearly enough was said about how they picked their samples. How you did it really matters a lot in interpreting results.
Using a bit of an exaggeration to demonstrate my point, if I say I collected 3 fish on my way from BC to Washington, you might conclude fish stocks collapsed. However, it also could have been that I drove from BC to WA, and I used a nalgene and a ham-sandwich to trap wherever I crossed a river. Methods matter.
• Now we're onto results. And the biggest, most mind blowing thing is... there's absolutely no discussion of the uncertainties involved. BLUH! The hell?!
If methods matter, uncertainties are astoundingly critical in these manners of analyses. 34 of 40 bottles contained pesticide residues. Really? How did you determine that? What constitutes 'presence?' To a non-analytical chemist, these questions might seem banal, but it /really matters/. There are no perfect methods that tell you exactly how much was contained within. Even your sampling screws you, and you need to make a statement about how much variability exists.
Example. Okay, let's say they detected 1 µg/L of compound X. What is the uncertainty on that measurement? If it's ±0.1 µg/L, you can probably conclude that there's actually some there, and the quantity is accurate. However, if the 95% confidence interval is ±2µg/L on the measurement... can you really say you have any at all? It's not distinguishable from zero when the error is around comparable to the measurement.
And this is when they give measurements at all! In some places, it's just 'was detected. My favourite was Pessac-Léognan Cru Classé, which contains 2µg/L pyrimethanil, and 5.8 µg/L procymidone, but (trace) fenhexamide. Well what the bloody hell does (trace) mean? Are we talking ng/L? pg/L? We saw a blip on the chromatogram, and we're calling it fenhexamide? Ugh!
They give no magnitude on most, just a discussion of 'it's an eeeeendocrine disruptor! Spooooooky!' Yes. It is. In clinical doses. But since they don't tell you what 'detected' means, never-mind actual amounts, you don't know if it comes anywhere close to significant doses.
Here's a panic inspiring fact: Your drinking water contains mercury. Quick, run for the hills! We're all going to die! No, not really, because the drinking water (hopefully) contains so little mercury that 'eh', it /really/ doesn't matter. I mean really, really doesn't matter.
Now, with the 'world famous wines', they actually give numbers, and if you're so motivated, you could hunt down what doses are required for effects. I'm not going to, because I have no way of knowing if those numbers are even imaginably accurate. That table is so half-assed that I couldn't help but notice that in Pessac-Léognan Cru Classé, pyrimethanil has only one sig-fig, whereas other mentions of it have at least two, or more (in fact, everything else on the table except (trace) has one trailing decimal).
You know what I'm going to do? I'm going to have a drink. I'm so underwhelemed by this black-box 'study' by a biased group that I'm resigning it to the junk-pile of studies, and taking my chances with some pesticide-laden ethanol. How about this one, instead? It's a review of numerous well vetted, peer-reviewed studies that suggest that wine drinking is beneficial to the health, in moderation. The mechanism, to my knowledge, remains unclear. Apparently these people didn't get the memo that the hundreds of pesticides are supposed to be harming their subjects.
In both cases, a group is making a claim, and one people might be inclined to accept. However, if you start poking at it, you realize that with the evidence provided, the story doesn't hold weight; just a little logical inference (and a pad of paper to do a dimensional analysis) makes the whole thing seem implausible.
But there seems to be some impression that skeptics are party poopers and cynics. Or worse still, know-it-all jerks. It's not. Skepticism insulates you from bad information, which can be more damaging that a lack of information. I only wish more people would but information through the grinder.