Re: About Quality (minini-essay) [BTPC Extension - PyramidWorkshop]

niels.froehling@xxxxxxxx wrote:


Hmm, I'd a discussion with my girl-friend, and explained her the
whole problematic with the quality-measurement by computer-
algorithms, and somehow realized that the PW-application
allready in a way helps to overcome the subjective problems
one may have with an algorithmic approach.

Okay, the biggest problem for an objective definition of quality
is that is has to be made by subjects (us) who's sense of quality
differs a lot. By using a test-group to tune or adjust a
setup of algorithms to, there is no evidence that it really have
happend [the desired effect]. Let's say 10000 people are testing 1000
pictures, then
the proposers _hope_ that the algorithm-setup would rate another
1000 the same way as if the 10000 people are going to do the test
instead.

We're now entering the field of psychology. No, it's not quite like that: If you run tests like this, you typically split your audience in two groups: The first one defines the quality metric, and the control group then checks whether the results are reasonable. Of course, you do not tell individuals which group they belong to, you just collect the results. From the data of the first group, the metric is calibrated, and the data from the second group shows then how well the calibration worked to predict the outcome of the control group.


But that assumption has no hard base. Also if the test
is repeated 100 years later, the sensibility of people in general
most sure have changed (more saturated pictures look better,
striped ones, ones with moire, etc.) so we end up with a different
quality-ranking.

Well. Sure. That's also why digital cameras from asian manufacturers often show over-saturated colors (German: "Bonbonfarben"). People there prefer it this way.

In effect that means, a lossy compressor sucks today and in 100
years it's un'beatable'? That for sure can't be right, because
something objective was looked for, and objectivity doesn't change
with time, that the Sun is hot (now) is unchangeable objective
truth, since ever, for ever.

I wouldn't be quite as pessimistic. The construction of the human eye will likely be the same in 100 years from now, and it will likely be unable to see certain frequencies in the same way as today. Thus, while you are of course correct that any "visual" quality measure might depend on the preferences of a specific observer group, one shouldn't be overly pessimistic: The metric will drift, but not from "excellent" to "useless".

Otherwise, you would deny that something like a useful general-purpose lossy image compressor exists at all, and each individual should use a compressor tuned to the viewing preferences. This is provably false, we still have JPEG-1 and most people are quite happy with it.

Also unanswered is the par-situation. Lets say compressor-A
removes hands from a picture of a croud of people, and compressor-B
removes the head. There is a semantic in the picture that an
algorithm never will realize, and never can be adjusted to. Under
circumstances the heads may be not important for the expression of
the picture, in others it's vice-versa. Let's say the algorithm
judges to be a par of the compressors, because there is a semantic
par; none of both, neither hands nor heads are in a way [more]
important.
This judgement of the par touches me - as esthetic sensing being [like
all of you]
immediatly, for me there don't exists a par [ever heard of
tristate-neurons?], only a humming
resonating thought-emotion chain trying to judge on it's own
and exiting the loop by: both compressors suck!

The problem is here that you compare images that are "too far apart" from each other. I don't think a quality metric is able to say much about this situation. It makes only sense to compare things that are already reasonable similar. ("What is the difference between a cookie and communism?" Well... does this make sense? Maybe not.)

So then quality is allowed to be subjective.

In effect every subject should get the abilities to tune/adjust
after it's own quality-metric, so as every subject is allowed
to setup gamma for the monitor; that adjusted gamma is only
valid for the combination person-A,monitor-B.

Maybe, but maybe you're putting too much load on the individual. Whereas for specific applications this tunablity would be certainly useful (natural images, medical images), a typical customer does *not* play with all settings of hers digital camera. (The 10%-90% rule: 10% of the features are used by 90% of the people). Thus, while theoretically the best concept, unapplicable.

While creating the PW-application I created a wizard-like interface
in that a center-pictures shows the actual choosen state and
surrounding pictures show different progressions, like changed
color-space, changed progressive-levels and changed quality.
In addition to the pure visual impact it's possible to look at
differencial pictures and distribution-curves to ease the doubts
if a settings doesn't eventually damages a picture too much.

That enables everyone to create his own definition of quality
within the framework, creating the combination person-A,picture-B.
So I got the idea that I allready solved the quality-measurement
problem on the subjective level, without having realized it. The
only thing missing for comparison is to display different
lossy systems, instead of only one system with various parameter
progressions.

So that's what I think about it: there will never be a metric of
objective quality, it would be simply wrong.

Any metric will be always wrong, except for the observer it was calibrated with, but I don't think that this observation helps. (Aka Lewis Carolls clock riddle: Which clock is better: The one that is one minute late per day, or the one that doesn't work at all. A: The second. It shows the precise time at least twice a day. Do you agree with Caroll?)


But that doesn't
mean that PSNR, MSE and other are useless; they are the objective
_helpers_ that a subject receives to analyze (hinterfragen) it's
own subjectivity, give ensurance in the 'correctness' of it's own
subjectivity, or leads to changes in it. And it allows to
change with the environment, the numbers may be absolute, the
interpretation is definitely _not_.

Well, sure. They aren't useless, but I think "objectivity" can be advanced by going beyond PSNR. Or to put it different: I believe that there are more useful quality metrics than PSNR that are still objective enough to be useful.

So long,
Thomas
.

Relevant Pages

  • Re: Non standard photometric interpretations
    ... converted to RGB using one of many different algorithms, ... time you are willing to spend on the conversion. ... you with a quality white painted picket fence. ... As to why we don't store the image as an RGB... ...
    (comp.protocols.dicom)
  • Re: The sickening reality of high ISO on a P&S
    ... I'm not attacking the quality of your ... All I'm saying is that given the same conditions, a dslr is capable of ... You've shown from your own experience as a p&s camera enhusiast. ... at some samples of pictures from the Sony A900 with one of the Carl ...
    (rec.photo.digital)
  • Re: Canon EOS 20D - Is this review fair?
    ... > underexposure, particularly with wideangle lenses." ... fairly easy and renders sharper pictures. ... > quality, but they also mean that users will have to put a bit more ... It might cost more but you'll ...
    (alt.photography)
  • Re: Different broadcast methods & perceived picture quality.
    ... Yes, well look at DAB, look at the now very poor dynamics on BBC FM, look at ... not be checked or even cared about in the quality sense unless its really ... see the quality of most HD pictures. ... channels has been lacklustre. ...
    (uk.tech.digital-tv)
  • Re: More one NASA management
    ... >of far lower quality than what there reporters had seen. ... Recce Office you'll know they keep their pictures quality tight to there ... The fact is that the atmosphere causes distortions - that is why the Hubble was ... challenging the theories is fine and that is how progress is made. ...
    (sci.crypt)
Loading