Baloney Detection

[tongodeon]

In "The Demon Haunted World" Carl Sagan wrote about the importance of having a good "baloney detection kit". A way to determine what claims are actually true and what isn't. Friends often come to me with various ideas and ask me what I think about them, and I usually start with a three step process:

1. What is the claim, exactly? "Ideas must be distinct before reason can act upon them." You have to understand a concept to agree or disagree with it. Is the claim coherent and explicable? Do we understand what sorts of issues or things are and aren't covered by the claim? Can we explain it back to someone who understands it, to demonstrate that we understand the claim?
2. Is it internally consistent? Is there any part of the claim that contradicts another part of the claim? Is a sales opportunity "not a pyramid scheme" which requires you to sell sales kits to your friends. Does a God say "thou shalt not kill" and then order His followers to kill great big piles of people?
3. Is it externally consistent? Does the claim jibe with what we already know? If it corrects our beliefs or assumptions, do those corrections check out?

This isn't the full baloney detection toolkit since there are lots of fine details like what constitutes "evidence", how to evaluate the strength of evidence, and what kind of fallacies to look for. And of course all knowledge is tentative, probablistic, and revisionary so anything that checks out might still need further modification in the future. But these are pretty good first steps in the road map. I've found that the most time-consuming (and the most often ignored) step is the first one. It can be really difficult to unwind and fully understand a perspective that's completely different than yours, but it's better than wasting your time on a straw man.