Precision vs. Accuracy and Significant Figures

One of my favorite news group is comp.risks. It's a moderated group where
they discuss risks associated with computing. Often, the topic is about how
results of some computer operation are presented, (mis)interpreted and acted
on by users. Because of the complexity of the information systems on
commercial aircraft comp.risks usually has insightful write-ups describing
how the systems involved failed. The SWA 1248 crash in Chicago was no
exception (see Risks Digest 24.15-20).

To refresh your memory, SWA 1248 crashed in Chicago in early December. The
pilot failed to apply the reverse thrusters in time and the plane over ran
the end of the runway, hitting a passing car and killing a young passenger.
According to the NTSB report, pilot used an 'On board laptop performance
computer' (OPC) to determine the landing distance. The OPC determined
distance for the plane to come to a stop was 560 feet from the end of the
runway; about a tenth of a mile. Given the parameters entered into the
system (speed, runway length, runway conditions? etc.) I wonder it is
possible to accurately compute a result to that precision.

This brings up the topic of precision. The mathematical definition of
precision from is, "The number of decimal places to which a
number is computed." Dividing 2.0 by 3.0 results in
..666666666666666666666666 and we are fat, dumb and happy because it is more
precise than .67. Java includes the method 'round()' in 'Math' to assist
the programmer in managing precision. Precision is not what we really want.

When dealing with measurements like weights, lengths and speeds, we are
really looking for accuracy. The mathematical definition of accuracy is,
"How close to the real value a measurement is." Accuracy is not managed by
simply rounding numbers. 'Significant Figures' are used to mange accuracy.

There are formal rules around implementing significant figures. When adding
or subtracting, the answer can contain no more decimal places than the least
accurate measurement. Therefore, when using significant figures, 235.2 +
..72353 = 236.9. When multiplying or dividing, the answer can contain no
more significant figures than the least accurate measurement. Therefore,
when using significant figures, 1.345 * 14 = 19.

If Java's API includes support for rounding, then why not support for
significant figures as well. Significant figures are a way of determining
the number of digits to which a number should be rounded. Rounding and
significant figures go hand in hand.

A quick Googling on 'Java Significant Figures' resulted in some classes that
allow for working with significant figures. The ones I looked at all have
some shortcomings.

The API's I've seen don't support exact numbers well. There are exactly 12
inches in a foot. There are exactly 16 ounces in a gallon. Exact numbers
have an infinite number of significant digits. So 1.234 feet is 14.808
inches. The answer is not 15 inches like the significant figure
multiplication rule would appear to require because there are exactly 12
inches in 1 foot.

A proper implementation of Significant Figures in Java would require a
change in the language definition. Granted, a number being 'exact' is the
default implementation in Java. In the context of significant figures, the
property of having an infinite number of significant figures is not formally
included as part of their definition.

Modifiers need to be added to the language to designate variables where the
number of significant figures needs consideration when computing the answer.

One way to designate significant figures could be:

int feetPerYard = 3 ; // an exact number
final int inchesPerFeet = 12 ; // an exact number
double significant distanceMeasured = 1.2373 ; //
track the number of sig figs in computations
double significant getSomeMeasurement() { ? } ;

The designation needs to be in the declaration so the compiler knows to
track the significant figures. This is a problem in loops:

while (test.hasMoretoTest()) {
count++ ;
total = total + test.readAnotherMeasurement() ;
average = total / count ;

While in the while block the number of significant figures need to be
tracked. If readAnotherMeasurement() returns {1.357, .3, 2.83}, then the
total has only 1 significant figure so the average is 1.

Note the programmer could screw up the calculations of significant figures
by initializing variables like:

double significant total = 0.0 ;

An initialization like the one above would set the significant figures for
any calculations to 2 which would mess up calculating the significant
figures for a set of values like {1.3579, 2.468, .1354}.

Implementing significant figures in Java will require the adding of a method
..significant(). The new method could be added to Math and StrictMath in
order to maintain consistency with .round(). The method could be called in
a similar fashion to round():

ave = var1 + var2 + var3 / 3 ;
sigAverage = Math.significant(ave) ;

The runtime environment would then need to check how the value of 'ave' is
computed and determine the significant figures of the values used to compute

Computers promised the ability to do more calculations with greater
accuracy. Accuracy was confused with precision and the result was simply
more zeros between the decimal point and the digit furthest to the right.

Math and science developed a process for maintaining accuracy during
calculations. The process is called 'significant figures' and their
implementation is formally defined.

Proper implementation of significant figures within a programming language
requires support during compile time as well as at runtime. To my
knowledge, no programming language includes built-in support for significant
figures. Significant figures are tedious and error prone to employ.
Computers excel at reducing errors in tedious, error prone processes
especially when the implementation is formally defined.

It is amazing significant figures haven't been incorporated in more
applications. I just checked MS Excel and it doesn't include a way to count
significant figures. Excel is designed to work with numbers. In
Microsoft's defense, I didn't see a way of computing significant figures in
Open Office either. The lack of support could be because significant
figures are not supported in programming languages. Ignoring the process
results produces deceptively precise results at the expense of accuracy.


Relevant Pages

  • Re: Precision vs. Accuracy and Significant Figures
    ... If Java's API includes support for rounding, ... Significant figures are a way of determining ... class MathMatrix extends MathVector { ... MathMatrix m = LorentzTransformation.generalizedTransform; ...
  • Re: How wrong can you be with your NPV and MIRR functions in EXCel
    ... "Harlan Grove" wrote in message ... I always build up those calculations using simple ... > almost always retain greater precision and accuracy than any ... rounded to three or four significant figures depending on the client) ...
  • Re: Excel Bug - Excel Geeks Unite!!
    ... Will round all values to cell format precision. ... Rounding to a specified number of significant figures does not necessarily solve the problem (see my previous decimal example involving ... That situation was potentially frustrating to serious numerical analysts, since it meant that there was far more precision available than you were allowed to use. ... Given the nature of this issue, I would tend to distrust any current package that does not exhibit the normal effects of finite precision binary approximation, unless the reason is clearly documented. ...
  • Floating point errors in finite difference calcs.
    ... significant figures of the H's to cancel out if H is order 1. ... left for usual double precision real variables. ...
  • Re: about c to fortran
    ... > more than 16 significant figures of precision. ... Where d is the number of decimal digits you write and ... no deficiencies and the other way is to make it so complicated ...