Re: ANN: FPOUT v2.0


"Marcel Hendrix" <mhx@xxxxxx> wrote in message news:24073516143561@xxxxxxxxxxxxxxxxx
"Ed" <nospam@xxxxxxxxxxx> writes Re: ANN: FPOUT v2.0

"Marcel Hendrix" <mhx@xxxxxx> wrote in message news:98271717143561@xxxxxxxxxxxxxxxxx
...
Why don't you publish an RfD to tighten up the ambiguity of
REPRESENT for negative string lengths?

I assume you're refering to "u" in the current definition of
REPRESENT (?)

Yes.

As it is unsigned "u" cannot be negative in any case. "u" is stated
to be the number of significant digits. "Rounding" and "significant
digits" are defined by mathematics. Consequently there are limits
to what one may validly round and to what number of significant
digits. No ambiguity there.

I guess I mistakenly thought you were proposing u=0 for rounding to an
integer. As 'u' is also the max. string length of the formatted result,
this is unfortunate.

I see there is still a good deal of confusion as what 'u' in the
definition really means. My understanding is as follows.

As 'u' is also the max. string length of the formatted result,
this is unfortunate.

'u' is *not* the max length of the string. It is the number of
"significant digits" - which has nothing to do with strings or
lengths.

Mathematics defines rounding as an operation which is applied
to a number and gives a numeric result. One cannot round
"non numbers" e.g. infinity. One can certainly not round a
number and get a string result.

To be mathematically consistent REPRESENT needs to be
interpreted as follows:

Take real number 'r' and round it to 'u' significant digits.
If 'r' was a number, take the numeric result and place its
ascii representation in the buffer, set flag2=true.
If 'r' is not a number e.g. NAN, INF etc. then set flag2=false
and place a user-defined string in the buffer.

The only ambiguity in the current definition of REPRESENT is
the lack of detail when flag2 is false (i.e. one cannot determine
the length and format of the string in the buffer).

It will be noted that "u" is never the length of a string. One may
however construe that 'u' characters will appear in the buffer
but that is true if - and only if - the rounding was legal in the
first place. Some cases of illegal rounding are:

- 'u' was not one or greater (maths does not recognize
"zero significant digits")
- 'r' was not a number and therefore can't be rounded e.g infinity
(maths regards infinity not as a number but as a "concept")

In the current definition of REPRESENT u=0 is illegal because
u is only defined to be the number of "significant digits".
However there is nothing to stop one expanding the definition
of u to include zero etc provided those values less than one
are not called "significant digits". There are advantages to taking
such an action and it is discussed in the document refered to in
the text of FPOUT.

Specifing -1 could be a solution. As you say, nobody
will specify a negative number of significant digits, so it can be
0 MINed and do what you want.

I have not seen a proposal for using negative values. Perhaps
you could explain what it does and give some examples?



.

Relevant Pages

  • Re: c-addr/len
    ... A large number of words use "c-add u" to indicate the address of a ... string and its length on the stack. ... referring to the number of characters or address units. ... When did "u most significant digits of ...
    (comp.lang.forth)
  • Re: ANN: FPOUT v2.0
    ... REPRESENT for negative string lengths? ... to be the number of significant digits. ... "Rounding" and "significant ... No ambiguity there. ...
    (comp.lang.forth)
  • Re: significant digits
    ... is two significant digits. ... As String ... Dim ZerosLeft As Integer, ZerosRight As Integer, ExtraZeros As Integer ... Debug.Print n, RSD ...
    (microsoft.public.excel.worksheet.functions)
  • Re: Format to n significant digits?
    ... will *round* a number to 2 significant digits, ... Dim PartsAs String ... code window that opened up. ... What does it do that the ROUND formula above doesn't do? ...
    (microsoft.public.excel)
  • Re: math rounding function needed
    ... a floating point into a string using the format command? ... If one wants rounding to a fixed decimal place, ... If one wants a number of significant digits, ... format command should always give you a rounding to the given number ...
    (comp.lang.tcl)