Re: A reasonable argument against double blind tests?


"j." <jason.burrows@xxxxxxxxx> wrote in message
news:1151135954.069230.226910@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Robert Morein wrote:
"j." <jason.burrows@xxxxxxxxx> wrote:
I've looked around a bit for this, and it seems it must have been
mentioned before - but I think there's a pretty compelling argument as
to why double blind tests aren't always the be all end all. (yeah,
this could get bad - I'm no expert

You're right - you're no expert.

Get lost.

Right, as I said I am no expert. I also said I'm sure this argument
must have come up before, but I was unable to locate the thread.

So obviously, there is a common counter augment to this - I'm just
trying to find out what it is.

If there is already a consensus on this, I'm just trying to determine
what it is.



This argument has been going on in audio newsgroups for at least fifteen
years.

There is a consensus that the tool can be useful for audiometric and
development work....so long as the effct being examined is identified and
users trained to hear it, and with the recognition that probably 50% of the
people will be hopeless and have to be thrown out of the test after the
training stage.

There is a consensus that most people can hear differences in audio level
when measuring with broadband white noise.

Beyond that there is no consensus.

Blind testing by itself does prevent a listener bias based on the knowledge
of which equipment is being testing. And this can influence results (a
sub-argument is always the difference between "can" and "does"). To this
degree it is scientific and pretty much beyond reproach. Advocates claim
that if an affect that is perceived when equipment is identified disapperars
under blind conditions, it is "proof" that the differences were imaginary.

The primary argument against comparative blind testing (particularly ABX
blind testing) is that it violates normal human musical listening and signal
processing so that it itself is an intervening variable....which is an
absolute "no..no" in test design. This is further heightened by the fact
that many attributes audiophiles hold dear, e.g. depth of image,
instrumental "air", etc. are to some degree artifacts concocted by the brain
to create the illusion of real music played in real space. The testing
seems particularly to destroy the ability to perceive this, according to
many. Advocates call it "imagination". Opponents call it "intervention".
Unfortunately this aspect of ear/brain processing has not been pinned down,
leaving the field wide-open to speculation.

Another facit of the argument involves "preference" versus "identity". Many
folks who are anti-ABX testing are not against blind testing per se (in the
sense of blind A-B preference testing) but oppose ABX testing because it is
a different kind of test...one that requires identification of differences,
a cognitive function versus preference, which is a holistic function
involving the emotions as well.. There has never been controlled, scientific
research done that correlates the information provided with the blind ABX
testing with other forms of testing (both blind and non-blind) in
determining whether differences that should be audible in audio euipment are
more or less readily perceived.

Accordingly, without definitive research the camps break down into stubborn
religiosity:

* the ABX camp holds that since ABX is a proven research tool for
audiometric research, it automatically becomes "the truth" and can be used
in anything audio...including the evaluation of equipment designed to
produce lifelike replication of music in the home, by untrained listener,
and open-ended evaluation (e.g. not knowing what they are listening for).

* the anti-ABX camp holds that the test itself interferes with the variable
under test, ie how a person hears/perceives/responds to music, and the
normal rise to consciousness of musical artifacts, and thus is an incorrect
instrument for equipment evaluation purposes.

Various means of attempting to research the issue have been rejected by one
side or the other, and thus it remains a matter of religion.

You might research the archives of RAHE for a long history of these issues.
Other forums contain substantial discussion as well, but oftern with more
noise/content ratio.


.

Relevant Pages

  • Re: A reasonable argument against double blind tests?
    ... to why double blind tests aren't always the be all end all. ... Blind testing by itself does prevent a listener bias based on the knowledge ... of which equipment is being testing. ... ABX can be done in the comfort of one's own home, ...
    (rec.audio.opinion)
  • Re: Ferstler in denial- he wont be at the DBT wake.
    ... Thanks again Harry for exposing ... ABX is not an identification test, ... comparison is from listening to music...it is a "left-brain" exercise. ... A simple blind "Which do you prefer, ...
    (rec.audio.opinion)
  • Re: SACD, DVD-A probably will not die
    ... In a blind taste test, CD-cola, DVD-cola, and SACD-cola ... Got an ABX Comparator? ... Aside from which an ABX Comparator is a standalone piece of solid-state ... Once you've demonstrated an audible difference, ...
    (rec.audio.opinion)
  • Theory: Why ABX doesnt work
    ... > Most blind tests seem to work that way. ... that ABX diminishes real differences. ... The ability of the brain to work ... participate in discrimination of abstractly labeled samples is that the full ...
    (rec.audio.opinion)
  • Re: Theory: Why ABX doesnt work
    ... >> Most blind tests seem to work that way. ... that ABX diminishes real differences. ... The ability of the brain to ... > participate in discrimination of abstractly labeled samples is that the ...
    (rec.audio.opinion)