Re: Floating Point Approximations.

Bob Badour wrote:
paul c wrote:

Bob Badour wrote:

...
The epsilon would be 16 times larger for BCD floats.
ie. 5e-7 versus 2^-25


Sorry, not hip to epsilon in this context. Does the above correspond with 50 millionths versus 32 millionths, or thereabouts?


Nope: 0.5 millionths versus 0.0298 millionths scaled by the exponent part of the float.

For floating point representations, epsilon is the rounding factor, which is half the least significant digit of the mantissa. If I recall correctly, a 32-bit BCD float has a 6 decimal digit mantissa with a 2 decimal digit exponent. A 32-bit binary float has a 24 bit mantissa with an 8 bit exponent.

Bloody Hell!

p
.

Relevant Pages

  • Re: Floating Point Approximations.
    ... The epsilon would be 16 times larger for BCD floats. ... Does the above correspond with 50 millionths versus 32 millionths, ... For floating point representations, epsilon is the rounding factor, which is half the least significant digit of the mantissa. ...
    (comp.databases.theory)
  • Re: Floating Point Approximations.
    ... The epsilon would be 16 times larger for BCD floats. ... For floating point representations, epsilon is the rounding factor, which is half the least significant digit of the mantissa. ... a 32-bit BCD float has a 6 decimal digit mantissa with a 2 decimal digit exponent. ...
    (comp.databases.theory)
  • Re: Floating Point Approximations.
    ... The epsilon would be 16 times larger for BCD floats. ... For floating point representations, epsilon is the rounding factor, which is half the least significant digit of the mantissa. ... a 32-bit BCD float has a 6 decimal digit mantissa with a 2 decimal digit exponent. ... I assume other BCD floats will be similar; although, I haven't studied them at all. ...
    (comp.databases.theory)