I'd Still Rather Have Bayesians Running Nuclear Reactors
SIPB talk today by Keith Winstein about "The Non-Conflict Between Bayesians and Frequentists" was interesting.
Basically, the two approaches to probability are as follows: Bayesians use prior probability (which may involve assumptions) to calculate the probability of a result given a new piece of data. Frequentists determine what answer they can give that will be right a given percentage of the time.
There's no real conflict, mathematically, it's just a matter of looking at different things. Bayesians look at the error rate among positive results, Frequentists look at the rate of false positives. Alternately, Bayesians focus on giving the best answer for every possible input, Frequentists focus on being most reliable overall for every possible output.
Less flatteringly: Bayesians use all of the information they have, but also some of the information they don't. Frequentists are reliable overall, but are willing to tell you manifestly untrue things so long as they only do so a small percentage of the time.
Frequentist vs. Bayesian is far from the only issue, which specific statistical test you use matters. Then again, the difference between reasonable statistical tests (even ones overly permissive for some values) might not be that significant in numerical terms, even if the difference can be quite substantial in financial terms.
Still, the conflict has some relevance to science. If you assume that people are studying a wide variety of hypotheses, only a tiny portion of which is likely to actually reflect the real world, then you might only have to admit a small percentage of false positives before the false positives overwhelm the true results.
Basically, the two approaches to probability are as follows: Bayesians use prior probability (which may involve assumptions) to calculate the probability of a result given a new piece of data. Frequentists determine what answer they can give that will be right a given percentage of the time.
There's no real conflict, mathematically, it's just a matter of looking at different things. Bayesians look at the error rate among positive results, Frequentists look at the rate of false positives. Alternately, Bayesians focus on giving the best answer for every possible input, Frequentists focus on being most reliable overall for every possible output.
Less flatteringly: Bayesians use all of the information they have, but also some of the information they don't. Frequentists are reliable overall, but are willing to tell you manifestly untrue things so long as they only do so a small percentage of the time.
Frequentist vs. Bayesian is far from the only issue, which specific statistical test you use matters. Then again, the difference between reasonable statistical tests (even ones overly permissive for some values) might not be that significant in numerical terms, even if the difference can be quite substantial in financial terms.
Still, the conflict has some relevance to science. If you assume that people are studying a wide variety of hypotheses, only a tiny portion of which is likely to actually reflect the real world, then you might only have to admit a small percentage of false positives before the false positives overwhelm the true results.