Re: exact p-values comparison

Thom wrote:
> > increases. So, as long as d_bar is not zero, you can get a significant
> > result just be increasing the sample size.
> > Bob
>
> Bob's post is spot on, but I'd like to expand on this one point. This
> assumes d-bar remains the same (hence Bob's careful use of "can, not
> "will"). If we increase N d-bar will nearly always change (usually a little,
> but sometimes a lot). I've occasionally seen this argument misused in
> manuscripts submitted for publication. For example, if the effect is large
> and sample size small I've seen authors argue that a non-significant effect
> is nevertheless important, without understanding that neither magnitude nor
> direction of effect is certain to remain constant in a larger sample.
>
> In your case confidence intervals might help you understand what's going on.
> As N increases the width of the CI decreases, so you should see two similar
> means measured with different degrees of precision (i.e., CI of different
> widths).
>
> Thom



Thank you, folks for your discussion.
I have 280 and 130 paired observation. Indeed, CI is wider for the
smaller sample.
I totally agree with the fact that p-value is only related to the
rejection/acception of null hypothesis and represents a statistical
aspect of importance of the results. However, sample estimates, such as
the mean of differences in my case, is clinical aspect of importance of
the results. One should not be used in place of another. They are
supposed to be used together. But some of my collegues argued that
p-values could be compared directly and alone. Even though I strongly
disagreed, I had to be sure. As Thom mentioned, indeed this is a common
misuse of test results in scientific publications especially medical
literature, but so is the comparison of p-values.

Bob, you're right, that next step would be to compare differences of
two sets.
BTW, what test would you use for that?
Although in my analysis the difference between differences is really
negligible (0.01) and I doubt that I would be able to get any
biological sense out of it even if it was statistically significant.
But anyway, for the future could you please let me know what test would
you use
for comparison of differences in means between two groups of paired
observations?

Thanks!
Anna

.

Relevant Pages

  • Re: Statistical Conventions in Social Science papers?
    ... of p-values. ... Every testbook you consulted is as Bob said made an error of ommission. ... Most textbooks assume that the reader knows what the domain is, ...
    (sci.stat.math)
  • Re: Tims survey and FSO
    ... Bob O`Bob wrote in ... >> how lazy I feel that day, who and how many are to be using the project, ... >> the importance of the code, ... Tsk tsk man. ...
    (microsoft.public.vb.general.discussion)