Re: What does the p-value mean



Murphy O'Brien wrote:

If p<0.05 the probability of being right when you say "P is not
Normal" is > 95%.

Well, not exactly. What I was trying to say was that the probability of incorrectly rejecting is a long-run property of the test procedure. It is not affected at all by a particular p-value. If, in a particular trial, your p-value is less than .05, then yes, your observed data fall in the rejection region, and that region has been defined to give the test procedure a 5% probability of false rejection. But that is not the same thing as what you said.


Some of this is angels on the head of a pin. If you reject the null when your p-value is less than .05, then you have a 5% probability of being wrong _if the null is indeed the truth_. That's a conditional probability, though, and you don't know in any particular case if the condition is true. I guess the 5% probability can also be thought of as a worst case unconditional probability too, so in that sense, the "> 95%" is correct.

- Peter
.



Relevant Pages

  • Re: goodness of fit ?
    ... Richard Ulrich wrote: ... For testing, if alpha> p-value, Ho is Rejected at the alpha level. ... to any SIGNIFICANCE LEVEL for false rejection at all! ...
    (sci.stat.math)
  • Re: Comparing fractions (or proportions)
    ... THE exact method provides a p-value which is BY ... result as least as extreme as that obtained by chance ... value i.e. the probability to get a data (x, n1, y, ...
    (sci.stat.math)
  • Re: Two nit-picks re definition of p-value (Was: goodness of fit ?)
    ... That's a nit pick on YOUR nit! ... If Ha is "greater than or equal", the more extreme in p-value is ... As I recall, in Fisher's Exact Test, the probability of the observed ...
    (sci.stat.math)
  • Re: Model Rejection
    ... Clearly if the probability of either class comes out below 2.5% or ... not allow rejection the combination of all of them still allows ... and a nondesign test set. ... with an accurate estimation of model parameters ...
    (sci.stat.math)
  • Re: False Discovery Rate
    ... any p-value is the probability of making a type I ... and your FDR would be 0%. ... >> of false discovery rate but I just don't get it. ...
    (sci.stat.math)