Re: D3/Linux and Flash Basic

"Tony Gravagno" <address.is.in.posts@xxxxxxxxxxxxxxxxxxxxxx> wrote in
message news:v1lq451ou8q75qk0ativl6p8gqujvbiobo@xxxxxxxxxx
I'll summarize up topside here: The problem Joseba describes isn't
with precision or decimals, it's the D3 F option that causes BASIC to
use floating point calculations. At least that's what he said right
at the very beginning. All the rest of this worldly knowledge about
how decimals don't belong in data, or how precision must be 4, is
simply hogwash.

You know Tony, it's hard to believe you actually mean this sort of thing.
One of the cardinal (but unwritten) rules of Pick data is that you never
store dates and numbers in external form. I guess we should be lucky that so
few novices are on this forum to be misled by this.

Chandru

"Peter McMurray" wrote:

Hi
As I suspected you are storing the data in external format.

Suspicions are very likely to be correct when they coincide with what
was explicitly stated in the original query.


Yhis is an absolute nono and explains the add issue.

It doesn't explain anything. We code with integers (internal format)
and use MR26 and similar masks out of convention. There is no rule
that says using decimals are a nono. Part of the reason for the
convention is that since MV is not strongly typed, mixing string and
numeric data can return weird results. But if you avoid the known
pitfalls there's no "absolute nono" about this.

On paper adding numbers with two decimals will never give you four.
One expects a computer to do the same. There's nothing intuitive or
obvious about the issue or Joseba wouldn't have asked it. If you can
explain WHY the issue is so readily explained, please do.


If you examine the data in more
detail I am quite sure that you will find items with 4 decial places
because
the data was not Iconved before storage.

He said in the original posting that the data has 2 decimals but math
is returning four. Even if some 4 decimal data were easily explained,
that doesn't explain the rest of the file being calculated to four
unrounded decimals. Haven't you read anything here?


CT is not a useful tool for this type of analysis you need to use DUMP.

*buzzer sounds again*
CT and DUMP show you exactly the same thing unless the period=decimal
is really an unprintable ASCII char.


Of course this does not work in the
FSI unfortunately so copy the file to a VME account and check there.

Or don't bother because CT, UP, ED, LIST, and other commands in the
FSI are fine.


PRECISION has a great deal to do with these issues so I can only assume
that
you have misread a blog.

He didn't say precision isn't related. This site requires
non-standard precision, but the rules for precision should still
apply. My blog helped to explain how BASIC works with different
precisions, beyond "PRECISION 4 is the default and changing it is
likely to cause a pain when other programmers are involved." Joseba
read the blog, you haven't. He didn't misread it. He read it and
he's continuing to experiment with options where he's now found an
anomaly. This is good information for everyone. Once again I think
you should consider this an opportunity to learn something instead of
perpetuating assumptions and misinformation.


I am not sure
about the F option and will read up on it as I never use it.

From D3 Ref: "When used with the o option, generates floating point
arithmetic."

As Joseba says, it doesn't seem to return expected results. I'm
curious what TL Support has to say.

As our other colleagues have suggested and I have agreed - it's best
not to use the option until it's confirmed to work properly.

T



.

Relevant Pages

  • Re: D3/Linux and Flash Basic
    ... with precision or decimals, it's the D3 F option that causes BASIC to ... use floating point calculations. ... you have misread a blog. ...
    (comp.databases.pick)
  • Re: Precision
    ... Whether one does the subsequent calculations in internal ie integer format ... I have always used external format with Precision 4 since my ... floating point calcs which are done in binary have around 3% error albeit at ... positions it is IMPOSIBLE to have a result of 4 decimals with the 2 ...
    (comp.databases.pick)
  • Re: Precision
    ... It's the same if we use Precision, floating point, or ... positions it is IMPOSIBLE to have a result of 4 decimals with the 2 ... May be something wrong in the floating ... remember what the associated actuarial calculations were. ...
    (comp.databases.pick)
  • Re: D3/Linux and Flash Basic
    ... The problem Joseba describes isn't ... with precision or decimals, it's the D3 F option that causes BASIC to ... use floating point calculations. ...
    (comp.databases.pick)
  • Re: Binary coded fixed point decimal
    ... If I want 3 digit of decimal precision then I have to adjust by 1000. ... Jean-Francois Michaud ... Anybody every play with the remainder when normalizing fixed-point ...
    (comp.lang.forth)