Significance testing is at best unsatisfying
On Deborah Mayo's Error Statistics Philosophy blog there is a discussion of a Bayesian attempt to ridicule frequentism:
‘Suppose we decide that the effect exists; that is, we reject [null hypothesis] H0. Surely, we must also reject probabilities conditional on H0, but then what was the logical justification for the decision? Orthodox logic saws off its own limb.’
In the blog comments, Paul Lawrence Hayes provides the original source, a talk given by Edwin T. Jaynes. His version appears on page 52 (pdf page 10):
"In the orthodox test, the sole basis for decision is probabilities conditional on the null hypothesis H0. Suppose, then, that we reject H0. Surely, we must also reject probabilities conditional on H0; but then what is the a posteriori justification for the decision? Orthodox logic saws off its own limb."
That context makes it clear that Jaynes is joking, and does not claim that this refutes frequentism. He even acknowledges that this is an acceptable proof by contradiction (albeit probabilistic), and states that even Bayesians will find occasion to use this form of reasoning.
But his complete talk, and even the title of the talk itself -- "The Intuitive Inadequacy of Classical Statistics" -- makes his larger point clear: we should prefer constructive proof to proof by contradiction.
But in the discussion on Mayo's blog, no one seems to understand this. Perhaps it is because of blind devotion of the discussants either to frequentism, or to Bayesianism?
‘Suppose we decide that the effect exists; that is, we reject [null hypothesis] H0. Surely, we must also reject probabilities conditional on H0, but then what was the logical justification for the decision? Orthodox logic saws off its own limb.’
In the blog comments, Paul Lawrence Hayes provides the original source, a talk given by Edwin T. Jaynes. His version appears on page 52 (pdf page 10):
"In the orthodox test, the sole basis for decision is probabilities conditional on the null hypothesis H0. Suppose, then, that we reject H0. Surely, we must also reject probabilities conditional on H0; but then what is the a posteriori justification for the decision? Orthodox logic saws off its own limb."
That context makes it clear that Jaynes is joking, and does not claim that this refutes frequentism. He even acknowledges that this is an acceptable proof by contradiction (albeit probabilistic), and states that even Bayesians will find occasion to use this form of reasoning.
But his complete talk, and even the title of the talk itself -- "The Intuitive Inadequacy of Classical Statistics" -- makes his larger point clear: we should prefer constructive proof to proof by contradiction.
But in the discussion on Mayo's blog, no one seems to understand this. Perhaps it is because of blind devotion of the discussants either to frequentism, or to Bayesianism?