Re: Interpretation of ANOVA when the entire population is under study

On 12 Dec 2005 00:18:23 -0800, "Sullivan2000" <jaitchis@xxxxxxxxxx>
wrote:

> Rich, thanks for the FPC references .. I am not sure how they bear on
> the problem at hand, but that may become clearer to me as my
> understanding of the problem unfolds.
>
> I suspect my problem is about asking the right questions, or
> alternatively, what can I reasonably conclude from the data at hand?
>
> Suppose I have just 1 months worth of complete data (all customers)
> from all stores and to make things simple assume that there are only 2
> stores .. not a sample of 2 stores but a universe of 2 stores.
>
> The data looks like
> store A store B
> N 1000 500 (N <> sample; N==all customers)
> mean 10 25
> sdev 5 5
>
> What can I reasonably conclude from this .. what reasonable
> generalizations might I make?

"Generalizations" is the key word.
If you are generalizing, you use the regular testing.

>
> a) I can conclude that store B sold more per customer than store A in
> that month. No generalization here, simply reportage. But
> "uninformative".
>
> b) Can I say that the observed differences are "unlikely to be due to
> chance?" .. what would this mean? The question being asked is
> tantamount to either

Yes, you say, "the observed differences are not chance."

That is a *gain*, presumably, over saying there are not
any differences except for chance. If you *know* that
one store sells twice as much because it is twice as big
with twice the staff, then it is silly to compare raw sales.
Presumably, you will compare numbers that can be
expected to be comparable.

Now, you know that the customers are different, the
locations are different, the stores are different, the
histories are different. There are a lot of things that
might be underlying whatever happens.

If the t-test (or whatever) says that they are "different",
then you can get interested in all those possibilities.
A test is an indicator. This is observational data, so it is a
shaky indicator, and says nothing about underlying causes,
or what matters among all the things that can be different.

If the t-test says that they seem, for now, to be the same,
then all those other differences apparently are either
trivial, or they are balancing out.

>
> b1) if customers were randomly assigned to stores (which we know is not
> true, but maybe is worth entertaining as a wedge to get a perspective
> on the results) then we "might have observed" a different result? A
> t-test would be appropriate here, no?
>
> b2) if we can regard the month as being a random sample of 1 from all
> months (and months being themselves interchangeable), then some other
> month is/is not likely to show a similar result
>
> It seems to me that there is information in the inter-customer
> variability that we can use to give sensible answers to these and
> similar questions.

Yes, there is some information in "the inter-customer
variability". Survey statisticians do not like to compare
City-1 with City-2; but the adjustments across sufficient
demographic categories *might* provide a believable test.

On the one hand, it is relatively easy to use a factor as a
covariate (race, sex, age); on the other hand, it is relatively
rare to use those factors to construct an error term that is
going to be robust. - I think that data-mining is going to
work on that latter problem in the next decade or two, but
I don't know of a quick answer right now.


>
> To do this, it seems to me that we have to perform a "trick" .. to say
> that the data that we have is really a sample from some loosely
> specified universe. And then proceed with the usual ANOVA and
> "significance tests". This seems defensible to me, or at least more
> productive than saying "what you have here is population data and you
> can only report it.. no generalization or sensible speculation possible
> .. a brick wall". So, what is the universe? perhaps - all purchases

I suppose it helps to read a lot of science fiction, and to
imagine all the parallel time-lines that are almost identical.

It is good to keep recognizing that the inference is shaky
and depends on assumptions. But this is *logic* and not
merely statistics. If you don't believe a comparison is useful
(because it is obvious, or because it is confounded) when
it is stated in English, you probably shouldn't be more impressed
when it is described with a test.


You are not going to re-play last year, or whatever the data
reflected. IF there is a difference, then you might consider
whether time plays a role, and if those "factors" that matter
are highly variable, or are changing. Standard customers?
- No difference, no big question jumps at you.


> over all future time (hence infinite and no FPC needed) for this set of
> stores (not seen as a sample of all possible stores).
>
> OK, is this sensible?

I

http://www.pitt.edu/~wpilib/index.html
.

Relevant Pages

  • Retailing at FRYs electronics
    ... In an era when "coddle your customers" is a retailing mantra, ... JOHN FRY built his first superstore smack in the middle of Silicon ... Since then Fry's has grown to 16 stores, ...
    (soc.culture.china)
  • Re: Amazed at Box Store Apathy (long)
    ... Generally I buy all serious purchaces from real tile distributors. ... I'm amazed at the apathy in these box stores. ... ignoring the customers. ... I guess as a little one-man business, ...
    (alt.home.repair)
  • Re: The ethics of returning cameras to the dealer.
    ... Then Bill should stop making his statements as absolutes. ... but a TREND if for customers to expect a lower price at larger ... three additional similar sales. ... How much discount will the stores get because of increased sales? ...
    (rec.photo.digital)
  • Amazed at Box Store Apathy (long)
    ... HD or Lowes to get a little something in the tile dept. ... I'm amazed at the apathy in these box stores. ... ignoring the customers. ... I guess as a little one-man business, ...
    (alt.home.repair)
  • Re: Collection Look Up - By Key in an Excel VBA App.
    ... once one stores an object in a ... to retrieve it. ... The Lookup I havent been able to compare, ... > since I havent been able to Look up an Obj by key from a collection. ...
    (microsoft.public.excel.programming)