Re: Precision
- From: jra <jrajoseba@xxxxxxxxx>
- Date: Thu, 9 Jul 2009 12:27:52 -0700 (PDT)
Only a comment. It's the same if we use Precision, floating point, or
a pencil. If I ADD numbers, and those numbers has only 2 decimal
positions it is IMPOSIBLE to have a result of 4 decimals with the 2
last ones different to 00. May be something wrong in the floating
point algorithm, may be something wrong in flash basic, or may be
other thing. But somewhere THERE IS SOMETHING WRONG.
joseba
On 5 jul, 04:27, "Peter McMurray" <excalibu...@xxxxxxxxxxx> wrote:
Hi
I noticed some furphys in some earlier posts on this subject so I decided to
do some testing on my own, as my practices/prejudices date back to
misadventures 32 years ago.
A couple of ground rules:
All my comments relate to common accounting practice in normal double-entry
book-keeping.
Decimal 4 is the generally accepted industry standard and the default in
D3 - by the way the D3 manual has an excellent write-up on this.
We do not use Galleons or Triganic Pus so all accounting uses decimal
currency first instigated by Peter The Great of course :-) I sometimes feel
that this is a pity as I have fond teenage memories of stirring up the comp
operators at work by beating them at adding up foolscap sheets of pounds,
shillings and pence - they would win the first pass but I would win on the
verification pass as I could isolate group patterns on the first pass and I
still remember 180 pence is fifteen bob.
It is a very long time since I last used Newton-Raphson and I cannot even
remember what the associated actuarial calculations were. If I needed it
again I would use floating point and isolate the programs. However I do
remember working on the Target audit almost 40 years ago and even then we
rounded the figures to $1000.00, so extreme accuracy at n decimal places is
not an issue in accounting.
One of the most common and time-wasting mistakes found in accounting
spreadsheets is people not using the ROUND function. If we have a
depreciation schedule for example and the tax accountant displays all the
individual calculations in currency format but just leaves the actual
formula to do its own thing then sums the column. The poor operator rattles
through the journal entry and wastes ages ticking off the entries because
her total does not add up to the spreadsheet total. Changing the format of
the columns would show all these trailing decimal places that have mucked up
the currency total, but how many normal operators would know how to do
that - hey it's printed it must be right.
The original Reality Pick Basic version 2.4d and 3.0 did not recognise the
overflow bit and happily returned a figure with the leading significant
digit missing. It also reduced the number of significant digits by every
point increase in the PRECISION statement - you had 15 to play with and that
was that. Change PRECISION 4 to 5 and you lost a leading significant digit.
Facts
PRECISION does not round a calculation it truncates it outside of the
floating point operations. The difference in my earlier examples between
PRECISION 4 and 6 demonstrate this.
Joseba got a final answer of 4 decimal places when using floating point
because he used floating point and the default PRECISION 4 for display
whereas the program was calculating a much larger figure. However he would
have had 9 places had he used PRECISION 9. The D3 manual gives the
excellent example of 8.0399999999999991 is the closest that INTEL/D3
floating point can get to 8.04.
ICONV rounds an answer witin the parameters of the PRECISION statement.
PRECISION 0 returns a value of 3 for PI = 22/7 rounded to MD4, PRECISION 4
returns a value of 3.1428 and PRECISION 6 returns a value of 3.1429 for this
reason.
PRECISION n truncates to n at each step of the operation. PI = 22/7*1000000
returns 3142800.
Every program that CALLS another program or shares a COMMON must use the
same PRECISION or suffer unexpected consequences. There may well be ways of
playing around with this but they are not standard procedure. .
MY Solutions that have worked well for 32 years.
Do not fiddle with PRECISION leave it at 4 as this is ideal for accounting
procedures.
Always store numbers using a standardised ICONV. In my situation dealing
with oil this is typically MD4 for pricing and MD2 for currency.
If you need 6 decimal places eg the silly Australian Tax department rule
then multiply by 100 and scale the answer back.
Always do calculations in external format and scale the answer as you go in
a long calculation. Remember we are going to round to currency anyway and a
normal person is going to check your invoices when they receive them with an
algebraic calculator set to decimal 2.
If you really do need Floating point then isolate the programs into a
separate package and document the devil out of them in the program. Do not
separate the documentation, it will not get updated and it will get lost,
and you will forget why you did that.
Anything non-standard will come back and bite you - DONT DO IT. First rule
of programming and dating - KISS :-)
Peter McMurray
.
- Follow-Ups:
- Re: Precision
- From: Peter McMurray
- Re: Precision
- From: BobJ
- Re: Precision
- References:
- Precision
- From: Peter McMurray
- Precision
- Prev by Date: Re: [ot] Passwords
- Next by Date: Re: Precision
- Previous by thread: Precision
- Next by thread: Re: Precision
- Index(es):
Relevant Pages
|
