HP30S numerical anomaly

I posted about a peculiar HP30S behavior over on MoHP, and I thought I
should post it here too.

Recently while looking at the calculator forensics web site,

http://www.rskey.org/~mwsebastian/miscprj/forensics.htm

I noticed that the HP30S was shown to pass the

arcsin(arccos(arctan(tan(cos(sin(9)))))

test with no error.

I wondered if this was really true, so I decided to look at all the
intermediate results during the calculation. I discovered that the zero
error result is the result of a peculiarity of the HP30S arithmetic.

The HP30S is apparently doing binary arithmetic internally, with 80 bit
precision (about 24 digit decimal precision). It can be verified that
ordinary operations such as +, -, * and / are giving 24 digit results. Some
of the other mathematical functions don't do so well. Joe Horn discovered a
while ago that square root only gives about 12 digit precision. The trig
functions can give substantially less than 24 digits; sometimes as low as
14 digits.

The full precision of a particular result can be seen using the technique
described on the calculator forensics web site to see guard digits:
(http://www.rskey.org/~mwsebastian/miscprj/guard_digits.htm).

The idea is to make a calculation and then subtract some of the leading
digits of the result and then you will see the rest of the digits. It may
be necessary after the subtraction to multiply by some power of 10 to shift
the remaining digits left so they can be seen.

The last calculation in the calculator forensics test is an arcsin, and
while looking at the details, I discovered the following peculiar behavior
of the HP30S.

In degrees mode, type the following into the HP30S (followed by enter):

Type arcsin(.156434465031) and see the result: 8.999999999

Type (arcsin(.156434465031)-8.999999999)*100000000000 and see: 46.45174843

The digits from the second calculation should be appended to the digits
from the first calculation to give a multi-digit result of:
8.9999999994645174843; the exactly correct result should be:
8.99999999946451748615554. In this case the HP30S gets 18 correct digits.

But now do the same calculation with a slightly greater argument:

Type arcsin(.156434465032) and see the result: 9

Type (arcsin(.156434465032)-8.999999999)*100000000000 and see: 100

The exact result *should* be: 8.99999999952252746476729

What happened? After some playing around, it seems that whenever the result
of a calculation (of a trig function, at least) is an integer followed by
..9999999995+, that is, the digits after the decimal point are nine 9's and
a 5 plus just a few more non-zero digits, the HP30S just rounds up to the
next integer. So what should have been:

8.99999999952252746476729

turns into 9.000000000000000000000000.

And the same thing happens on the other side of an integer (slightly
greater than some integer). For example:

arcsin(.156434465049) returns 9.000000001:

(arsin(.156434465049)-9)*100000000000 returns 50.86970993

Realizing that the first result was actually 9.0000000005+some more digits
(it got rounded up to 9.000000001 in the display), we can construct the
full result of 9.0000000005086970993. The exact result *should* be:
9.00000000050869710116852--the HP30S gets about 16 correct digits.

But now try arcsin(.156434465048) and get 9

Furthermore, (arsin(.156434465048)-9)*100000000000 returns 0
, indicating that there are no more digits after the integer 9.

Again the digits after the decimal point have been dropped, and we don't
even get 12 digit accuracy.

It appears that whenever the digits after the decimal point are greater
than .9999999995 or less than .0000000005, they get dropped--at least as a
result of a trig calculation; I haven't checked all other caclulations, but
this may happen always.

Why do they do this? Probably to avoid some of the apparent small errors
that happen in simple calculations, which perplex naive calculator users.
For example, do (sqrt(2))^2 on a typical calculator and get
1.99999999999...(depending on how many digits the calculator carries, and
whether there are hidden guard digits) instead of exactly 2. Or do sin(pi)
and get something other than zero. Or do (6^-1)^-1 and get 5.99999999999...
instead of exactly 6. The HP30S gets the exact expected result, but at the
price of reduced accuracy when a result is near an integer.

So finally, this is why the HP30S gets exactly 9.00000000000000000 when
running the calculator forensics test. To see the real error, one should
subtract the input argument of the test from the final result. That is, do:

arcsin(arcos(arctan(tan(cos(sin(9))))))-9

On the HP30S, the result is 0, leading one to believe that the HP30S is
*really* accurate. But redo the test as:

arcsin(arcos(arctan(tan(cos(sin(9.1))))))-9.1

to foil its rounding near integers and see a result: 4.833288903E-11

Not all that great for a calculator that can do 24 digit calculations, eh?
.

Relevant Pages

  • Re: .99999... still=/= 1
    ... > A scientific calculator does approximate convergence with a certain ... > number significant digits. ... i.e. any partial sum, or do you mean the whole series, which is defined ... The expression you wrote with it stopping after the mth term ...
    (sci.math)
  • Re: Thatll Teach Em - Long Division
    ... if only so you are not totally stumped if your calculator fails. ... I remember a teacher when I was at school in the 1970s whose standard ... punishment for any misdemeanour was "cubes". ... boys to call out digits (the number of digits depended on the severity of ...
    (uk.media.tv.misc)
  • Re: FAQ 4.2 Why is int() broken?
    ... is much of an issue - 8 or 12 digits are enough. ... For example on my HP-48 calculator. ... can display 0.999999999999 exactly, ...
    (comp.lang.perl.misc)
  • Re: Math errors in python
    ... They don't even share three digits beyond the decimal point. ... > yet not all fractions can be represented by decimals. ... Hardware designers have settled on a binary representation of floating ... And BTW, your calculator is not, in general, more accurate than the ...
    (comp.lang.python)