Re: Record demagnetizers

In article <ggnpj3l2vj0slqcfr1cc48uj4ai90fr0ig@xxxxxxx>, Signal
<nobody@xxxxxxxxxxx> wrote:
Jim Lesurf <jcgl@xxxxxxxxxxxxxxxxxx> wrote:

Well I don't like to draw direct analogies between drug and hifi
blind tests (which are far from equivalent), but yes the underlying
principle is nice in theory. However, pertaining to controversial
areas of hifi, there are practically no DBTs of a grade which could
be called conclusive.

There I would agree with you. Most audio DBTs are poorly designed and
inadequately conducted. Which is why the audio snake-oil merchants
still make a good living.

OK, lets take as an example the test reported a month or so ago in the
JAES. Would you say this was poorly designed, etc? if so, can you
perhaps explain in detail, and say how you would run a better test to
deal with the problems you see? I'm curious about this as I've already
seen it dismissed in one magazine.

Sorry Jim, haven't read the JAES article. I do know that details on the
equipment used was withheld by the authors for fear of being "nitpicked
to death". That seems like a mistake.

They have now put up a page listing more details of equipment, etc. FWIW in
practice issues like this would normally be left for a peer referee to pick
up. Although I don't know the details of the JAES process, so am assuming
normal academic journal methods.

So I can see their point that - since it wasn't - they were happy with what
was published. I can also see why they had the worry, but that said, I also
was curious to know all the relevant details. The details show they used
various players, venues, etc. So it would be difficult to dismiss the
results on the basis that 'player X' or 'amp Y' wasn't up to the task.

So far as I can tell from reading audio mags, they generally don't
report any details of controlled tests, and when they are mentioned
dismiss them with no assessable explanations. Seems to be a mix of
"ignore completely" and "dismiss because we know we can hear what we
say we can".

Sceptics often follow the same sort of herd mentality outside of the
scientific establishment. The Oohashi study for instance received a
battering, and is dismissed 'out of hand' by most sceptics.

By definition "sceptics" may tend to do that. :-) However it is fair
enough to have doubts. The key then is to be able to design and carry out a
test which would tackle the point of contention used as the basis for
doubt. Or link the work to other work to see if there is a clash or
inconsistency.

A distinction here is that the Oohashi work was interesting, but in a field
where almost no other similar tests/experiments have been done and gave
supporting results. Whereas in the general field of 'comparison listening
tests' a number have been carried out, using various methods and protocols,
and they generally point to conclusions similar to the recent JAES article
report.

Thus in the Oohashi case a sceptic would be doubting what is essentially a
single experimental study which gave a contentious result. Whereas with the
topic of what people vaguely call "DBT" there have been many such tests, of
various kinds, with broadly consistent results which fit in with our
general knowledge of the relevant areas of science. So it becomes difficult
to use one case as an analogue for the other.

Hence I doubt most readers will be aware of them, or be in a position
to judge if they were well run or not. Thus they have to rely upon what
the magazines tell them. Given this, it seems to me quite
understandable that readers accept what the 'experts' in the mags tell
them, and which often seems consistent with their own
experience/impressions.

Perhaps, but how many readers fit this scenerio? In my experience, most
people are capable of arriving at their own decisions and using critical
thinking.

Yes, people often arrive at their own decisions. The question in each case,
though, is if they are making a simple error. :-) Alas, people believe
all kinds of things, often contradicting each other and clashing with
established scientific understanding and evidence. If they reach a decision
based on unreliable evidence, or being mislead, or having relevant data
withheld, then no matter how intelligent they are, they can still reach a
false conclusion.

To some extent here, in audio, I suspect there is a parallel with something
I've often noticed in undergrads. They have picked up from school the idea
that an 'error' means a 'mistake', and is something to be ashamed of. The
feeling seems to be that it implies personal failing and deserves
criticism. Thus is an embarassment to be avoided at all costs.

Yet 'error' is a vital part of learning *if* you can recognise that errors
exist. One of the most powerful ways to learn is to be able to spot
possible errors, and then do a better job by examining something in a way
that tackles the possible source to see if a mistake was made.

This, in science, is a vital part of experimental design. Thus the idea of
test methods which aim to run tests where the decision is based on the
sound, and to either suppress, common-mode, or randomise and deal
statistically, with possible sources of error. This is why you end up doing
an 'error analysis'. Not to show the conclusion is absolutely reliable, but
to assess how reliable it may be.

Unfortunately, many in audio tend to fall back on variants of, "I know I
can hear differences so any tests that look to me to imply otherwise must
be wrong!" and then refuse to engage in any real test that might show if
they were correct or not. Thus we have no way other than faith to decide if
they are right about what *they* claim *they* can do. This seems to me to
help no-one at all. So denying the possibility of honest error may block
progress and obscure the truth. What might have been an honest mistake
becomes something more problematic. The result is to generate emotional
argument as people become more concerned to 'defend their ideas' than to
try and understand what is actually going on.

My personal suspicion is that they might in some cases be right, and in
other cases may be perceiving a 'difference' but assuming the wrong
'cause'. All too easy if the comparision isn't done with due care, and
knowing the possible sources of error. But when people refuse to put their
claims to a critical test, we can only really guess as they prevent us from
gathering any assessable evidence.

Aside:

In the case of the JAES paper the specific counfounding factor actually
agrees with evidence I (and others) have noted in other contexts. That
SACD/DVD-A 'high rez' recordings may simple be 'better made' than much of
what appears on CD, in ways which have no real technical relevance to
SACD/DVD-A/CD as such. So the same recordings 'downsampled' onto CD
would also sound good. If you look at work on the audiomisc site comparing
different sources you can see that modern CDs are often level compressed or
clipped when there is no reason for this to *have* to be the case so far as
CD as a medium is concerned. Done for 'artistic' or 'sales' reasons.

The real problem may be that many CDs are cynically made on the basis that
"louder sells" whereas DVD-As and SACDs may be more likey to be made on a
"quality sells" basis. But this refers to the content, not the inherent
abilitiy of the media. However, if we *are* mislead into assuming that
SACD/DVD-A is the 'answer' then we may be mistaking the container for the
contained. In due course, the same makers will then start dumbing down the
content by over-compression and clipping[1] so we are no better off, and
have wasted money duplicating recordings.

This, for me, is the real shame of a lot of what people argue about wrt
'tests' and the claims in magazines, etc. It takes people's eyes off the
ball. We end up arguing about what may be pointless and irrelevant whilst
the music companies get away with demonstrable problems like clipping the
recordings or level-compressing them to death. I'd prefer all the publicity
and argument to be directed to such targets which stand some chance of
leading to an improved result. But I guess people prefer to argue about
tests whose results they dislike. :-)

End of aside:

So, if someone thinks the tests that have been done are 'wrong' then the
relevant course is for them to design and engage in alternatives that deal
with both the well-known 'confusing' factors, and any others they have in
mind. We could then decide on the basis of the evidence - including the new
evidence they produce. Otherwise we just have their assertions of faith and
claims to compare with the test evidence. In this context, my personal view
is that I have doubts about what someone claims if they refuse to engage in
a test whose outcome would provide evidence to support/refute what they
claim.

Personally, I am quite happy with the results of many of the variants of
the 'tests' since they do tend to agree with my experience. However I'd be
quite happy to accept that others *can* hear what they claim, but would
rather like some evidence to assess, and then reach a decision on that
basis, not just on having to rely upon claims. This does not seem
unreasonable to me, but no doubt some would disagree. :-)

Slainte,

Jim

[1] OK, SACD is nominally unclippable as the modulator can go unstable if
you try to go to loud. But I suspect that means we'll just end up with some
other garbaged SACDs if the system survives and grows - which seems
unlikely.

--
Electronics http://www.st-and.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Audio Misc http://www.audiomisc.co.uk/index.html
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
.