There are a few good explanations already (also less good and very bad) so I give a simple example:
You throw a coin five times and I predict the result correctly each time.
#1 You say that I have precognition powers, because the probability that I don’t is less than 5%
#2 You say that I have precognition powers, because if I didn’t the probability that I would have got the outcomes right is less than 5%
#2 is a bad logical conclusion but it’s based on the right interpretation (while #1 is completely wrong): it’s more likely that I was lucky because precognition is very implausible to start with.
jonahx
Dead on again.
What this and your other comment make clear is that once you start talking about the probability that X is true, especially in the context of hypothesis testing, you've moved (usually unwittingly) into a Bayesian framing, and you better make your priors explicit.
You throw a coin five times and I predict the result correctly each time.
#1 You say that I have precognition powers, because the probability that I don’t is less than 5%
#2 You say that I have precognition powers, because if I didn’t the probability that I would have got the outcomes right is less than 5%
#2 is a bad logical conclusion but it’s based on the right interpretation (while #1 is completely wrong): it’s more likely that I was lucky because precognition is very implausible to start with.