Re: Blind Cable Test at CES

On Sat, 19 Jan 2008 20:29:21 -0600, corbett@xxxxxxxxxxxx (John
Corbett) wrote:

In article <4_GdnarYYPMIqw_anZ2dnUVZ_jydnZ2d@xxxxxxxxxxx>, "Harry Lavo"
<hlavo@xxxxxxxxxxx> wrote:


I know a great deal about statistics, particularly their practical use,
although I am not a statistician.

Well, I am a statistician.
You seem to be so confused about statistics that you can neither perform
the calculations nor understand what they mean.


Before looking at your calculations, we need to consider something that's
been overlooked so far in this thread.

These calculations apply in the situation where the number of correct
answers has a binomial distribution. You---and other posters---seem to
have assumed that is appropriate in this example. However, a binomial
distribution describes the number of successes in a *fixed* number of
independent and identically-distributed binary trials. That would appear
to be the case if we believed that the experimenter planned to do 39
trials. I suspect that he did not pick that number before starting his
testing. Without knowing the stopping rule, we really cannot calculate
values needed to do proper tests. For instance, if he planned to do tests
until he had 15 wrong answers, and it happened to take 39 trials, then an
ordinary binomial distribution would *not* apply---we would use a negative
binomial distribution in that case. If he planned to go until he got 24
correct ansers, and that happened to take 39 trials, we'd use a different
negative binomial distribution. If he had some other rule (e.g., "stop at
8 pm"), yet another distribution would be called for. If we don't know the
stopping rule, then we ought to be very cautious applying the usual simple
procedures.

So, assuming the binomial model is applicable, let's check some
calculations ...



So I gave myself a refresher course. Used the normal table as
recommended by the book to estimate distribution for samples 30-100
(below that the "t" table),

I really doubt that your book says to use "t" for a *binomial* problem.
The t distribution is needed when you have independent estimates of the
mean and variance, but the variance is a function of the mean for a
binomial distribution, so the t is not appropriate. If you are using a
normal approximation for a binomal distribution, you should use a "z"
table, no matter what the sample size is. (Of course, that assumes the
sample is large enough to use a normal approximation in the first place.)


used the formulas to calculate the standard error of
"r" (in this case, .04876),

What is "r" here, and what formulas did you use to get .04876?


double-checked my work, and still come out with the 95% confidence level at
23.22 (1.96 standard deviations).

95% _is_ a confidence level; 23.22 is _not_ a confidence level.
(Do you know what a confidence level is?)

1.96 standard deviations would apply if you were forming a two-sided
confidence interval, or if you were performing a two-tailed test; it is
the z value correpnding to an upper tail area of .025. Since you are
looking at a one-tailed test here, you need the cutoff z value for an
upper tail area of .05, which is 1.645.



In other words, 24 of 39 *is* significant at the 95% level, just as I had
previously stated, according to my calculations. Even if you want to use
two standard deviations, it works out to 23.3.

No.

If you use the normal approximation to the binomial, and use the
continuity correction, you should get that 24 of 39 has a p-value of
0.1001. If you don't use the continuity correction, you get .07477;
that's what Oliver Costich did. If you do the exact calculation, based
directly on the binomial instead of an approximating normal, you get
.0998; you can see that the properly-used normal approximation is very
good. So 24 of 39 is *not* significant at the .05 level, although it is
significant at the .10 level. (Elsewhere you have indicated that you're
okay with significance at the 10% level.)

Earler in this thread, you wrote

The fact is, there is nothing magical about 95%, except that it has been
widely accepted in the scientific community to meet their standards of
"probably so".

Yes, 95% is widely accepted primarily because it is widely accepted. ;-)

It gives odds of 19:1 that the null hypothesis is invalid.

No, that is NOT what it means.

In hypothesis testing, calculations are done
_under_the_assumption_that_the_null_hypothesis_is_true_.

Stop and read that again. Got it yet?

P-values and significance levels involve probabilities of results that may
occur if the null hypothesis is true; they are not probabilities that the
null hypothesis was true in the first place. (BTW, 95% is a typical
*confidence* level. Hypothesis tests involve *significance* levels, and
.05 is a commonly used value. Although there is a connection between
these concepts, it is _not_ as simple as saying that 95% confidence is the
same as 5% significance.)

Doing real statistics is not about just doing arithmetic as an excuse to
avoid having to think about the data.

Agreed. I was trying to keep it as simple by using ordinary binomial
methods (which make the calculations easy with a TI calculator).


.

Relevant Pages

  • Re: Blind Cable Test at CES
    ... Well, I am a statistician. ... Before looking at your calculations, we need to consider something that's ... binomial distribution in that case. ... sample is large enough to use a normal approximation in the first place.) ...
    (rec.audio.opinion)
  • Re: Pre-arranged NABC championship deals evidences
    ... A person describing himself as no statistician, ... just making arbitrary calculations that have no justification at all." ... likelihood ratios as if they are a valid measure of suspiciousness, ... The poster speaks once more: ...
    (rec.games.bridge)
  • Re: Need Help Determining the "True" Mean of a Sample
    ... > I'm a software engineer, not a statistician, so please forgive my ... > The distribution for these samples is such that about 75% of the ... only 4% as much of the total variance. ... of dropping immediately to zero when the N is under 58. ...
    (sci.stat.math)
  • Re: CI on ratio of two slopes
    ... >statistician), the ratio of two normal variates is Cauchy, which has ... >would like to provide an interval estimate. ... covariances are correct and ends up with an exact distribution which is ...
    (sci.stat.math)
  • Re: Riddle me this
    ... > My data does not fit a normal distribution. ... I'm not a statistician. ... I you really mean that you want it to be "statistically meaningful" then you'll ... any hint as to whether the value is intended to be an absolute cut-off value, ...
    (comp.lang.java.programmer)