Re: New version of John Hayes' tester

Anton Ertl wrote:
> "David N. Williams" <williams@xxxxxxxxx> writes:
>> Anton Ertl wrote:
>>> "David N. Williams" <williams@xxxxxxxxx> writes:
>>>> And indeed the point about small x and y is that the relative
>>>> comparison becomes meaningless when they are the same order of
>>>> magnitude as the tolerance, eps. For example,
>>>>
>>>> 1.2 * eps - 1.0 * eps .2
>>>> ---------------------- = ---- ,
>>>> 1.2 * eps + 1.0 * eps 2.2
>>>>
>>>> which is not so small.
>>> If I understand you correctly, your two numbers are 1.2*eps and
>>> 1.0*eps, and eps is the (absolute value of the) r3 parameter for F~.
>>> Right?
>> Right.
>>
>>> Then what you compare is whether
>>>
>>> |1.2*eps - 1.0*eps| < eps * (|1.2*eps| + |1.0*eps|)
>>>
>>> No, relative comparison is not meaningless when they are the same
>>> order of magnitude as the tolerance, absolute comparison is:
>>>
>>> |1.2*eps - 1.0*eps| < eps
>> Doesn't that depend on whether you expected the test to pass or
>> fail? In this case the relative test fails for eps not that
>> small (e.g., .2) and the absolute test passes.
>
> Yes. I would not consider 1.2*x to be approximately equal to 1.0*x
> for any x; or if I would consider them approximately equal, I would
> consider them approximately equal for any x.

Touché, if you're saying that when 1.2*x is relatively
approximately equal to 1.0*x for one x, it's relatively
approximately equal for all nonzero x. :-)

I agree that the above example is not so good. Let me try
another approach. The comparison

|x - y| < eps * (|x| + |y|)

is bound to fail for fixed eps less than unity if |x|/|y| is
either small enough or large enough. One can say, well of
course, they're not relatively nearly equal!

But zero is special -- there's no way to say that a nonzero x is
relatively near to y = 0 with the above inequality, for eps < 1.

When you compare, say, two algorithms for computing a function,
it can be that relative comparison is both useful when the
function is not close to zero, and not what's wanted when it's
absolutely close to zero.

In complex-tests.fs, there are tests of both kinds on a small
real or imaginary part of a function, i.e., there are
comparisons to exact zero, and comparisons to a "gauge" function
where a different algorithm computes a different approximation
to zero.

I've reviewed this in complex-test.fs, using ttester and each of
the three choices: one or the other of the relative and
absolute comparisons turned off, or both on. There I'm not
really interested in great accuracy, but it's nice to know that
it's reasonably okay.

Right away, I found it inconvenient that the exact test at the
start of FNEARLY= distinguishes between signed and unsigned zero
when ABS-NEAR is set to zero. So I changed

F2DUP FEXACTLY= IF F2DROP TRUE EXIT THEN

to

F2DUP F= IF F2DROP TRUE EXIT THEN

The nonstandard word F= works as expected in both pfe and gforth
on my system. I'd say it's the equivalent of F- F0=, except
that ANS94 seems ambiguous about whether F0= correctly ignores
the sign of zero.

From now on in this message, the sign of zero is to be ignored.

The above F= test is superfluous unless ABS-NEAR is set to zero.
Should it be simply removed, even when ABS-NEAR is zero?

Argument to keep it: although relative comparison when x or y
is zero is not appropriate, such values might slip in, and one
probably wants equality to satisfy NEARLY= when absolute
comparison is turned off.

Argument to remove it: the user might better be warned by
failure of the relative test in such cases, and then make his
own choice whether to make the absolute tolerance nonzero.

I favor keeping it, because I can't think of a situation where x
= y and relative comparison fails in which I wouldn't want to
regard x and y as nearly equal. But my experience with floating
point numerical analysis is not uniformly deep. Opinions,
anybody?

The following summarizes the smallest nonzero choices for the
tolerances on my system for complex-test.fs with complex.fs.
(same for pfe and gforth), assuming only powers of 10:

i. All tests pass except for some comparisons near zero:
REL-NEAR = 1E-15 ABS-NEAR = 0E0

ii. All tests pass:
REL-NEAR = 0E0 ABS-NEAR = 1E-13

iii. All tests pass:
REL-NEAR = 1E-15 ABS-NEAR = 1E-15

Although not identical, results with the complex-kahan.fs and
pfe implementations of the complex lexicon are similar. When
ABS-NEAR is zero, REL-NEAR has to be set to 10E in order for all
tests to pass.

>>> If you test a numerically unstable algorithm with the wrong
>>> parameters, relative matching may need the REL-NEAR parameter
>>> increased, but one should not use such parameters in real life, and
>>> probably not in testing (at least in this kind of testing), either.
>> Indeed. But I did have in mind that one might want to use the
>> tests for numerical algorithms, not just for the formal
>> correctness of word implementations.
>
> I think that relative approximate comparison is especially appropriate
> in that context, for the reasons given above.

I think so, too, modulo the singularity at zero. For example,
variants of relative comparison are used in most of Cody's
celefunt tests. A basic variation with complex numbers is to
use a relative comparison defined with the complex number
absolute value instead of the real number absolute value, but
that's another story.

> For testing standard words, one might want to test ill-conditioned
> cases like PI FSIN, and for that absolute approximate comparison has
> its uses.

In the case of complex-test.fs, I had thought it convenient to
do both comparisons, the relative one first, for when the
compared numbers are not supposed to approximate zero, and the
absolute one second, for when they are supposed to approximate
zero with possibly incommensurate scales, and the relative test
fails.

But after the review, I'm finding I want to know which of the
two comparisons fails in a given test, so I can see if
discrepancies among the three implementions indicate
possibilities for improvement. Now I'm wondering if FNEARLY=
shouldn't be vectored:

VARIABLE FNEAR-XT

: FNEARLY= FNEAR-XT @ EXECUTE ;

: FREL= ( F: X Y -- S: FLAG )
(
LEAVE TRUE IF THE TWO FLOATS ARE NUMERICALLY EQUAL, OR
RELATIVELY EQUAL BASED ON THE TOLERANCE STORED IN ABS-NEAR.
)
F2DUP F= IF F2DROP TRUE EXIT THEN
REL-NEAR F@ FNEGATE F~ ;

' FREL= FNEAR-XT !
\ ' FABS= FNEAR-XT !

Then FREL= and FABS= could be made available with ttester and
the user could supply whatever else he wanted to revector
FNEARLY=.

Maybe that's getting too complicated for a basic tester.

This message is already long enough, so I'm posting more
reactions to ttester in yet another message...

-- David
.

Relevant Pages

  • Re: Absolute vs Incremental 3D programming
    ... 18i control for which we do not have a post processor which means I'm in ... anything but absolute, but have done manual programming in incremental. ... day long and the largest deviation from zero will still be in the .015in ... I'll also mention that the CNC control does math in AT LEAST six place ...
    (alt.machines.cnc)
  • Re: Absolute vs Incremental 3D programming
    ... Fanuc 18i control for which we do not have a post processor which ... never posted 3D tool paths in anything but absolute, ... deviation from zero will still be in the .015in window. ... I'll also mention that the CNC control does math in AT LEAST six ...
    (alt.machines.cnc)
  • Re: Absolute vs Incremental 3D programming
    ... anything but absolute, but have done manual programming in incremental. ... deviation from zero will still be in the .015in window. ... I'll also mention that the CNC control does math in AT LEAST six place ... All math is done from machine zero, no matter how many work zeros ...
    (alt.machines.cnc)
  • Re: Gunbuster 2
    ... if you've got a "physical canceller," that is... ... "Below absolute zero", therefore, ... Phillip Thorne wrote further: ...
    (rec.arts.anime.misc)
  • Universe Opposed In Triplicate
    ... Zero and one are base2. ... Tell me where and how there can be "expansion" without an absolute ... Absolute neutrality. ... dimension is our infinity dimensioned dimension. ...
    (sci.physics)