Re: Consumer Reports digital camera reviews

-hh wrote:
"Mark²" <mjmorgan(lowest even number here)@cox..net> wrote:
Bill Funk wrote:

Their reliability ratings are scientifically unreliable, for an
excellent reason: the sample is skewed to CR readers who bother to
respond.

But that is true of every scientifically sound poll...every
survey...and every process of review...beta...drug study.

Not true. There's good and bad ways to conduct samplings. CR's
methods are known to be one of the bad (statistically flawed) methods.

Based upon what research?

Look, guys... I make zero claims to be a statistical guru. But their
findings regarding reliability are about as straight forward as it gets.
Subjective opinions and "feel" are not involved when relating the number of
failures you've paid for concerning your vehicle or lawnmower. The "image
quality" of a camera often IS subjective, and can be horribly scewed
depending on the eye, experience, and purpose behind the reviewer. I have
real problems with that. But people here love to take it to the heights in
EVERY area, and I believe that's unnecessay. Might someone falsify a
report? Sure. But logic should tell you that those paying for information
via subscription are NOT particularly likely to sabatage their own
subscription's usefulness by concocting or concealing repairs. People who
subscribe to CR tend to do so because they are NOT pre-tied to a particular
brand.

I think many go after CR because their crummy reviews of (for example)
cameras and printers are poor at best...at least for those who are serious
about imaging. Half the time, they pull ridiculous boners...like a huge
group of point & shoots...with only one DSLR. We all agree it's pointess.
But all the jumping and assuming about the most basic reports of needed
repairs is just too much. The naysayers lose credebility when they cut such
wide swaths.

The sample has to be willing to participate...and...the only
answers considered are for products owned by the participants.

Unfortunately, that's only half the story beacuse the participants can
choose to opt-out. See:
http://en.wikipedia.org/wiki/Self-selection

The variable that messes up the statistics is: what was the
recipients' underlying motivation that resulted in them either tossing
or responding?

Right. Just like many polls. Just like many drug studies...where people
fear they may be subject to arrest--should, for example, and study reveal
they are (ex) drug abusers...etc.

Very simplistically, response likelihood varies with individual
motivation.

Sure. But it can work in both directions, which means it isn't as
destructive to the sample as is implied.

Factors that influence motivation include an "axe to grind", as well
as their perception of where they personally are at in the product's
lifecycle (people pay attention more if they're close to buying, etc).


Nothing new or weird there.

Unfortunately, that's all too true: most people don't realize how
easily self-selection will cause a statistical bias in your data.

This is why I don't claim they are without bias. The reason I mentioned
other studies is because while people shoot down CR based on variability,
they accept other studies that are similarly compromised.

What CR has going for it is a complete lack of advertisers. Could
they still be biased? Of course!

I dropped my subscription in disgust years ago when they praised one
automobile to high heaven for having "X" as a safety enhancement, and
then declaired the idential feature "X" to be a dangerous hazard to
avoid in another auto brand's review within a half dozen issues. This
wasn't the first gross self-contradiction I have ever found in their
writings, but merely the nail in the coffin. I've read them
periodically since then and I have seen no evidence that they have
changed.

But the alternative is magazine titles that are
entirely beholden to adverts for their existence.

It may sound appealing in theory, but a known consistant bias is
actually eaiser to find, which makes it easier to then compensate for.

Road and track does "reviews," but they accept ads from direct
rivals...within the same issue! They are kissing everyone's arse...which
can't be particularly helpful.

I don't really think disagree on much... It's a matter of degree, and some
of the more appropriate bases for determining how strong an opinion or
statement is warranted.

-
Images (Plus Snaps & Grabs) by Mark² at:
www.pbase.com/markuson


.

Relevant Pages

  • Re: LuxAsm questions
    ... > Beth, when timing stuff, you should discard all but the ... another point about statistics: ... it was NOT a case that a "disk access" had just happened ... there's a _REASON_ and direct _BENEFIT_ for doing so... ...
    (alt.lang.asm)
  • Re: Truck driver shot and killed in Tampa
    ... report criminal activity that is an example of racism. ... but the reasons people commit crimes does have relevance. ... There is a certain consistency between crime statistics among ... predisposed and in evidence without a cause or reason for those ...
    (misc.transport.trucking)
  • Re: Which Is Better?
    ... The reason they gave was from a DBA perpspective like it was ... MERGE really necessary here - do you do any inserts in that loop, ... statistics for both approaches and compare them to see what could ... Did you capture any other statistics aside from run time ...
    (comp.databases.oracle.misc)
  • Re: Power Law of Probability? (The book "Chances Are" by the Kaplans)
    ... W. Watson wrote: ... So who gave it "good reviews". ... It's an interesting book for its background, philosphy and history ... writing their stories than writing about probability and statistics. ...
    (sci.stat.math)
  • Re: Power Law of Probability? (The book "Chances Are" by the Kaplans)
    ... So who gave it "good reviews". ... It's an interesting book for its background, philosphy and history ... writing their stories than writing about probability and statistics. ... So, if the authors had read Steve's historical papers, they could have ...
    (sci.stat.math)