Ruby Numbers
Hi I'm new to ruby.
Started with learning-ruby.
In first_steps it looks at using ruby as a calculator
1 + 2
=> 3
3 / 2
=> 1
3.0 / 2
=> 1.5
all this I understand BUT
4.1 % 2
=> 0.0999999999999996
Should be .1
and whats more there has only been one decimal place used so the answer
should have only 1 decimal place.
another
3.945 * 2.62
=> 10.3359 4 decimals should only be three
Dave
.
Relevant Pages
- Re: Ruby Numbers
... In first_steps it looks at using ruby as a calculator ... are running into here is how Ruby handles floating point values. ... => 10.3359 4 decimals should only be three ...
(comp.lang.ruby) - Re: Math errors in python
... They don't even share three digits beyond the decimal point. ... > yet not all fractions can be represented by decimals. ... Hardware designers have settled on a binary representation of floating ... And BTW, your calculator is not, in general, more accurate than the ...
(comp.lang.python) - Re: calculating PI
... ' PI calculator ... When I assigned the expression to PIE at the end and printed PI-PIE, ... I'm trying to get the source-code to a fellow TI'ers program that calculates ... first 5200 decimals of Pi in 184 hours of continuous operation using ...
(comp.lang.basic.misc) - Re: ((.99999.... == 1) == 1) = 1
... >Using Powertoys Calculator on Windows XP, I can disply up to 140 decimal ... as many digits as you want. ... 140 come no where near an infinite number of decimals. ...
(sci.math) - Calculator Answer Doesnt Match Excel Answer
... calculator, and got $106,000,000. ... Excel gives me $103,246,753.25 as the answer. ... as currency with two decimals. ... college-level Finance class exam. ...
(microsoft.public.excel.misc) |
|
