Re: math error, 1/3+100 etc., Lotus 1-2-3 Rls 9

The math is right, you are seeing the impact of unavoidable approximations to your inputs.

The IEEE standard double precision (used by almost all computer software, Windows calculator being a notable exception) approximation to 100+1/3 is
100.3333333333333285963817615993320941925048828125
and when you subtract 100 from that underlying value, you should get
0.333333333333329
You can see more of the underlying value by calculating
100+1/3-100.333251953125
which should be
0.0000813802083285963817615993320941925048828125
and will display as
0.0000813802083285964

Since some 16 digit numbers cannot be approximated in binary, Excel's documented behavior is to only display 15 digits, followed by zeros if more figures are requested. Try help for "specifications" (or similar key words), and see if Lotus also documents only displaying 15 digits.

Jerry

Nick wrote:

I typed the following into a new Lotus spread*** (Release 9 for Win,
old but wondering if also true for new):

1/3+100

I got the following answer:

100.333333333333000

The trailing zeros, of course, are wrong. By contrast, Calculator
(Win95a), when 1/3 is entered, answers with: 0.3333333333333

In response to
100+(1/3)-100
Lotus yields: 0.333333333333329

When cell A2 is
1/3+100
yielding 100.333333333333000
then when another cell is
+A2-100
the answer is:
0.333333333333329

The Lotus cells were formatted for Fixed number format with 15
decimals, the max in the spin control.

When the formula is "1/3", the answer is close enough to correct (i.e.,
0.333333333333333). Adding "+100" either during the first entering of
the formula or in a subsequent amendment to the cell's formula is when
the answer gets thrown out of whack.

When the formula is

+A4+100

(cell A4 already containing 0.333333333333333)

the answer is

100.333333333333000

Again Lotus is wrong, and in the same way.

Is this a bug in Lotus spreadsheets? Is it already known and has it
been subsequently fixed? Or is the bug elsewhere in my system? I tested
two platforms:
-- One processor is a Pentium 233MHz with MMX; it appears free of
the well-known Pentium floating-point bug (tested in Calculator with
962306957033/11010046=87402.62820273 and in 1-2-3 with
962306957033/11010046 yielding 87402.628202734100000 (correct except I
don't know if the least significant zeros belong in the correct
answer), the test being from Scott Mueller's Upgrading and Repairing
PCs (16th ed.)). The system board is a First International Computers
PA-2011. Win95a is running.
-- The other is a Pentium II 266MHz Dell running Win98SE.

These machines are not on the Internet; one's sometimes networked to a
laptop which, at other times, is exposed to the Internet, but it has an
auto-updated firewall, auto-updated antiviral, and other security.

I don't have newer Lotus 1-2-3 programs, so I can't tell if they have
the same issue.

Anyone know?

-- Nick



.