Re: Question for Alan Shank...or others

Does not a specified confidence level imply a definition of an unfair
game?

You can set up the test this way. So if we go back to our guy who walks
up to us and asks us if he was cheated the following question seems
fair:

1) I have a thousand trials. I want the confidence interval to be 99%.
How far off from expectation do I have to be to fail the test?

But that isn't the math people are doing in this thread. They are
starting with the 4% and finding the easiest test for that to fail.
That *isn't* fair.

But to get back to my looking forward vs looking back idea, we still
have a big problem. Here's a looking forward example:

We suspect a dice table to be crooked. We record the next 2000
outcomes. We perform some statistical test, and it fails at the 99.9%
confidence interval. What does that mean?

Well, it is bascially saying that a result like that will occur on a
fair table only one time in 1000. Either we just happened by freak
chance to be at the table on a freak occurance, or the table is
crooked. Be very suspicious.

Now let's take a look at a looking backward example. Let's assume you
are a gaming commission representative in Vegas. Somebody comes to you
who just got off a craps table where he had the worst result of his
life. He happened to record every outcome from the past 2000 outcomes
and asks you to perform a statistical test. You perform the same test
as in the looking forward problem, and sure enough it fails. Should you
be very suspicious?

Well, not necessarily. There are a ton of craps tables in Vegas. Every
day or so, one of those tables is going to go ice cold. If we just look
at results from tables where people looking backwards and suspect
something was wrong, we aren't looking at a random trial. We are
looking at a highly biased trial. So the assumptions behind our typcial
statistical testing are violated.

Now, there are statistical tests we can do. If we look at a set of data
where we lost more than expected, we can find unusual patterns that
make us suspicious. But the simple fact that we are looking backwards
at a cold table does not allow us to draw conclusions from typical
statistical tests.

.