Real world question

This is a barn door question.

We recently completed a salary survey of association members. About
10,000 member names were drawn at random (using SPSS) from the entire
member database. These were invited via e-mail to take the web-based
survey. About 2000 took the survey, about 1700 answered the salary
question.

I have already done the t-test and know that respondents who gave
salary info are a different population than those who did not answer
the question.

Here's the set-up for the big question: The sample, the total
respondent base, and the respondents who gave salary information all
have roughly the same profile. When I say roughly I mean within 1%
(usually tighter) on such criteria as degree held, age, geography. If I
put on my marketing hat, where the standards are a bit looser, I'm
satisifed that the same population, more or less, is involved. (I leave
aside for the moment the fact that there is demonstrably different
behavior within the overall respondent group.) In fact, as a marketer
I'd probably feel justified making a large budget commitment based on
what amounts to close to a 5% response rate.

But the statistician-wanna-be in me is concerned about the methodology.
This is the real-world; what we're doing is closer to making data
sausage than identifying generalizable truths. (For the record, though,
two independent salary surveys touching, in part, the same population
and conducted using more traditional methods, i.e. drawing the sample
and seeking to maximize response, achieved results within $1,000-$2,000
of annual compensation to our findings at all experience levels.)
Still, we would like to have some justification for our confidence.
(Other than the sheer bravado of being marketing folks.)

So, did we just get lucky or is there some basis for our method
working?

Opinions are welcome; citations would be helpful (there's a faculty
advisor on a master's thesis lurking about).

Thanks.

.

Relevant Pages

  • Sample vs Population Question
    ... We recently completed a salary survey of association members. ... 10,000 member names were drawn at random from the entire ... I have already done the t-test and know that respondents who gave ... But the statistician-wanna-be in me is concerned about the methodology. ...
    (sci.stat.edu)
  • Re: SF Publishing Opportunity
    ... op publishing, but of presentation. ... The majority of respondents seem ... know it came from the passion stated above. ... perhaps someday becoming an accepted member. ...
    (rec.arts.sf.composition)
  • Re: SF Publishing Opportunity
    ... op publishing, but of presentation. ... The majority of respondents seem ... know it came from the passion stated above. ... perhaps someday becoming an accepted member. ...
    (rec.arts.sf.composition)
  • Re: Sample vs Population Question
    ... > We recently completed a salary survey of association members. ... > member database. ... > I have already done the t-test and know that respondents who gave ... What did you do a t-test on that you can conclude that the ~ 300 people ...
    (sci.stat.edu)