Re: Which? Audio tests

In article <doclof$kad$1@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>, Rob
<removethispatchoulian@xxxxxxxxx> wrote:
> I wrote to them recently concerning their testing procedure for audio
> electronics. I don't subscribe, although I occasionally leaf through
> library copies when I'm supposed to be working :-)

> In answer to the Q: do you use blind testing and matched levels, she
> said 'yes'. However, she went on to say that it's not always needed
> because, as they're completely independent, there is nothing to gain
> from recommending any particular model.

When she says, "not always needed", does she mean, "but we don't always
bother to use blind methods"? Or does she mean, "We always do that, but
feel it isn't always strictly needed?" It seems an ambiguous answer to me.

The magazine/association may be 'independent', but the individuals coming
to the tests may have their own individual pre-conceptions. Thus one of the
aims of 'blind' methods is to try and ensure they don't bring these to bear
for any reasons other than those which are intended to be relevant.



> I thnk this is generally fine, and nice of them to respond. It's also
> rather vague - I think I'll reply asking them to publish the detail of
> their tests (level matching, respondent selection, room, other kit,
> music and medium used), and suggest that they think about:

FWIW I tend to regard any test or measurement methods whose details are
unspecified as being of dubious value. If you don't know exactly how the
results were obtained, how can you assess their reliability?


> I think that would do me - I don't really want to know about cables, and
> while some technical data (power output maybe) might be useful, I don't
> think they have the expertise to measure/interpret the information.
> Just knowing that people can reliably and qualitatively differentiate
> is a pretty good start IMO.

One snag is that people may give 'reasons' for 'differentiating' which are
actually incorrect. This may mean that their conclusions may not apply in
your or my case as the report implies. Without sufficient details and a
cautious approach this may reduce or negate the value of the reports...

FWIW I still recall an old 'Which?" report which essentially told those who
had no prior understanding that the Quad ESL57 and the Goodmans Maxim
sounded the same and were similar in use... The problem here is an
over-simplification of the findings or characteristics which may mislead
the innocent.

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Audio Misc http://www.st-and.demon.co.uk/AudioMisc/index.html
Armstrong Audio http://www.st-and.demon.co.uk/Audio/armstrong.html
Barbirolli Soc. http://www.st-and.demon.co.uk/JBSoc/JBSoc.html
.

Relevant Pages

  • Re: A reasonable argument against double blind tests?
    ... have with certain attributes of high-end, "serious" audio. ... Even a total novice, with no electronics training at all, ... I don't blame the DBT protocol per se, I useit in my profession, ... so they will perceive the sound as being better. ...
    (rec.audio.opinion)
  • Re: A reasonable argument against double blind tests?
    ... one does not need to be an "authority on electronics" to see the ... silliness we have with certain attributes of high-end, "serious" audio. ... does sound different from the proverbial "straight line with gain" ideal ... differences in amp sound sighted that they miss when listening blind. ...
    (rec.audio.opinion)
  • Re: A reasonable argument against double blind tests?
    ... However, one does not need to be an "authority on electronics" to see the silliness we have with certain attributes of high-end, "serious" audio. ... Even a total novice, with no electronics training at all, should be able to do a decent level-matched, and hopefully blind listening comparison to see the preposterousness of what some of the local tweakos are saying. ... Now, as I recall, you work with amplifying gear that sometimes actually does sound different from the proverbial "straight line with gain" ideal amplifier performance. ...
    (rec.audio.opinion)
  • Re: Sony DTTV recorder
    ... I decided to do a 'straw poll' of my current undergrad electronics ... have thought younger people would not notice the noise. ... over substance culture in the design dept. ... Presumably in the black fronted audio model range, ...
    (uk.tech.digital-tv)
  • Re: Audio Techno-Voyeurism Exposed
    ... I studied electronics for four years at an appropriate institution here in ... working, as my primary source of income, as an audio tech, servicing ... brands of audio equipment, since 1976. ...
    (rec.audio.opinion)
Loading