Re: Very weird resolution issue. Bug ????? Seriously !!

Sung Soo Kim wrote:
About '+' operator. Actually, Peter pointed out that my memory was actually right, that
a+b+c is (a+b)+c, not a+(b+c). But it might be changed in certain circumstances
(like a gray area that standard cannot step in that someone mentioned above, like 80bit FP?).
If you have an complex enough example that ISO committee can disagree on the behavior of
a+b+c, please let me know. I'm curious.

Just introduce parallelism at the machine instruction level. ISO C90 and ISO C99
are written assuming that (short of interrupts and operations on volatiles)
that at the abstract machine does one operation at a time; the standards contain no
information on how to map that into the reality of modern processors, -except-
through the guarantees provided at sequence points. Guarantees which are routinely
broken if any kind of optimization is used.


If something is clearly defined, and if the result does not match with that, then we call it
a bug. What do you call it?

Me? I tend to call it one of two things:

A) Something that the person only *thought* was clearly defined, but which falls through
one of the many holes or constraints in the C standards; or

B) "Realpolitik"
http://en.wikipedia.org/wiki/Realpolitik#Example_of_Realpolitik_in_Politics_in_the_United_States

"Realpolitik" is why the Intel x86 CISC architecture still has the greatest share of
the non-embedded market, long after Intel and AMD internally recoded their
architectures to be RISC (or other non-CISC) underneath the hood. "Realpolitik" is
living with what you get, providing that the results are not provably wrong.


Let me phrase that more strongly: if you write a C program that involves
floating point operations, and the C program conforms to every last
constraint clause in C, and you run the same executable twice on the same
machine and you have no random number generation happening, then as far as
C is concerned, it is absolutely and completely legal for the same executable
to produce different answers on different runs! In C, floating point calculations
are not required to be deterministic -- only to be accurate within certain limits.


And physically, if the codes are the same and all environments are the same and the
compiler is the same, and even no random number generator is used, then how can the
result ever vary? Your assumption on the situation is so deterministic that I can't
imagine anything like what you mentioned. Where is the dice? That behavior may be legal
that but that's unrealistic.

Interrupts have been known, in production architectures, to change the FP state in
subtle ways.

Even for the language's legality issue, the spec says FP result depends on platform
(and/or precision) and that's the only accepted variation on FP '+' operation.

No, no, you over-estimate the specification! I have the C89 spec right here (well
worn!) and what it says about FP result accuracy for the arithemetic operations and
for the <math.h> library, is... NOTHING. Except, that is, that it places some
restrictions on the legal values that can result if a floating point constant appears
in the source and is in the representable range but is not exactly representable; and
it place similar (but not identical) restrictions when a value is being converted
to or from a different floating (or integer) type. But if you want a guarantee
that (say) 1.0 + 1.0 is within 1 ULP of 2.0, you will not find it in C89!

That said, representation of FP is not C language's responsibility. Right?

C89 2.2.4.2.2 defines a *model* of how floating point values are represented in
C, and various functions such as frexp() are allowed to assume that arithmetic
in C happens *as if* that model is used. C99 devotes an appendix to the IEEE 754
floating point model, and specifically allows implementations to #define a particular
constant if the implementation's floating point meets all clauses of the model
(which isn't always the case: DSPs have a tendency to use something that is *almost*
IEEE 754 but not to support all of the rounding modes that the 754 model requires.)
C99 says that if that particular constant is #define'd by the implementation, then
there are a series of library functions that shall be available that have certain
relatively well-behaved properties. But support for that floating point model
is strictly optional.

Well, it is your freedom to interpret that as 'the compiler can change what the machine
spit out as it wish within a certain amount of error', but I'll rather interpret that
'C compiler must not touch what the machine spit out'.

You would be wrong. The C language does not define how computer architectures must
work: it defines how certain operations must behave -regardless of how complex
the machine code is that is needed to achieve that behaviour-. There *are* real
C libraries that "fix up" the result of some machine operations, especially when it
comes to the edge-case behaviours that the C language defines as resulting in
particular return values.

And if the machine spit out different results on simple equations like 'a+b' across runs,
then it is a useless computer.

Or the computer is implementing "round alternately" (perhaps not the official term.)
When have a sequence of additions or subtractions and always round 1/2 ULP "up" or
always round 1/2 ULP "down", then the consistency of the rounding leads to a cumulative
error that would not be present if you rounded 1/2 ULP randomly; even rounding
1/2 ULP up one time and down the next time has better statistical properties.
"round to even" is the usual modern tactic taken to control this issue, but it isn't
the only possibility. The point is when you examine the matter from a longer viewpoint,
having a+b always come out the same way is not necessarily the best!

So... I don't think I have to change my opinion: The heart of computation and scientific
software is 'consistency' (of course accuracy as well).

For floating point calculations, 'consistency' lost out to 'speed' at least 15 years ago.
And no, I am not exaggerating just to make a point.

I work in a non-University research organization. Our sub-division has about 200 people.
Number of official mathematicians on staff: one (a statistician). Number of other staff
with math degrees: as far as I know, one (me, a programmer). Number of times in the
last 15 years I have been asked to do a numeric analysis or to otherwise ensure that
the program was getting the most accurate answer it could: zero (but I insisted on doing
some anyhow). Number of the incoming "young turks" with snazzy modern object-oriented
computer science degrees that I have ever heard even *mention* undertaking a numeric
analysis for their projects: zero. Number of researchers who want to use the clusters
or want multi-core processors on their desktops... ur, was that carry three heads and
four knees, or carry four heads and three knees? Beyond any easy counting, whatever it is :(
.

Relevant Pages

  • Re: Problem With Round Function
    ... I read where they are improving accuracy of floating point ... "Ronald W. Roberts" wrote: ... >> I'm having a problem understanding the Round function. ... >> Num DecPt Answer ...
    (microsoft.public.dotnet.languages.vb)
  • Re: floating point display problem
    ... public class PrintTest { ... Floating point numbers aren't exact. ... far from guaranteed that a 7 decimals number can be represented ... different from x to also round to y. ...
    (comp.lang.java.programmer)
  • Tough problem...
    ... I am hoping somebody here has enough expirience ... in math and floating point round off errors, ...
    (borland.public.delphi.language.basm)
  • Re: Rounding
    ... > Now you gets incorrectly round down. ... Never ever use floatingpoint arithmeting when it is money you ... I already limited the number of lines with floating point math to 1, ...
    (comp.os.os2.programmer.misc)
  • Re: Float comparison
    ... you about floating point arithmetic. ... so in future please do NOT call your preferred round the common rounding and do NOT base your arguments on the assumption that it is the rounding method used. ... There are unusual situations in which something else is better and this is acknowledged by the C standard and the floating point standards. ...
    (comp.lang.c)