Re: Confidence Intervals for Bridge Simulations (long)

On Oct 31, 1:42 am, Charles Brenner <cbren...@xxxxxxxxxxxx> wrote:
On Oct 30, 9:54 pm, paulh...@xxxxxxxx wrote:

(4) Calculate the lower and upper bounds of the confidence interval, p minus or plus three times ssd

In warning against careless interpretation of simulation results
without taking into account the realities of sampling statistics,
beware of overstepping in the opposite direction.

The approximation that all probabilities within a 3 standard deviation
confidence interval are equally probable and outcomes outside that
range are impossible -- that's cookbook statistics.

More trials do give more reliable conclusions. But since the effect is
obviously gradual, it must be that even small differences -- so-called
"statistically insignificant" differences -- among small numbers of
trials mean something. If after a grand total of one simulation 3NT
looks better than 2NT, that's evidence that 3NT is really better.
Grant me some sort of prior probability distribution assumption as to
how good each contract might be, and I can compute the strength of
that evidence. (And without some such assumption, confidence intervals
imply nothing.)

Charles

Of course all outcomes within the confidence interval are not equally
probable, the sample mean is the single most likely value for the
population mean, and values toward the edge of the interval are far
less likely. My point is that posters are habitually acting as though
they can draw conclusions from a simulation of 100 hands, by looking
only at the sample mean, ignoring variability. I don't think such
simulations give evidence of any practical value.

As for your requirement of an assumption about the prior probability
distribution, doesn't the Central Limit Theorem allow us to infer the
population mean from a sample without knowing the population
distribution? And make a statement about the strength of the evidence?

.

Relevant Pages

  • Re: The discussions on confidence intervals
    ... > you only need to elicit one probability from the doctor. ... The doctor has to gather evidence and diagnostics until she's is able ... be aware that there are *FOUR* kinds of confidence intervals: ...
    (sci.stat.math)
  • Re: The discussions on confidence intervals
    ... > you only need to elicit one probability from the doctor. ... The doctor has to gather evidence and diagnostics until she's is able ... be aware that there are *FOUR* kinds of confidence intervals: ...
    (sci.stat.edu)
  • Re: computer simulation
    ... computer simulation, what would YOU do? ... Not if you can't provide evidence for your hypothesis and a way to ... If we were to learn more about the probability ...
    (talk.origins)
  • Re: Evolution increases the computational ability of organisms.
    ... we are inside a purposeful simulation or not. ... understanding the universe. ... And "absense of evidence" tells a sane person what ... by your speculation that at some ...
    (talk.origins)
  • Re: Confidence Intervals for Bridge Simulations (long)
    ... analyze bridge hands. ... simulation and reported the results in terms of how often 2NT, 3NT, 3S ... most players, double-dummy analysis is useful for part-score and game ... In medicine confidence intervals of 95% are common. ...
    (rec.games.bridge)