I do have problems using ceil(), floor() or round().
I tried to create a method to round to a certain number of decimal places. Most of the time it works (I solved it like described in the help to ceil() ) - but sometimes it just does not work as expected.
Return dResult
[/code]
For wert = 0,6333333333333333 and stellen = 5 the situation is as follows:
dResult = wert*Potenz => 63333,3333333333284827
dResult = Floor(dResult) => gives 63333 as expected but the next line
dResult = dResult/Potenz => gives 0,6333299999999999 // in most cases I do get the expected 5 digits on the right side - but not every time.
Can anybody give me a hint what I’m doing wrong? Thank you!
Nothing - a double simply cannot, in this case, represent .633333 accurately enough.
It all comes down to how to represent that value as the sum of powers of 2.
Every integer can be expressed as some sum of 2^0, 2^1, and so on.
The fractional portion in a double is also powers of 2 as 2^-1 (1/2), 2^-2 (1/4), etc.
And if you try to generate .633333 you’ll find the best you can do is get VERY close - but not exactly that value.
see Floating-point arithmetic - Wikipedia
The good news is, you can force it to display properly to your users so they won’t know that anything is “wrong”, and that value will always be represented internally the same way, so a comparison will work. In other words:
p = 10. ^ 5.
d1 = 0.6333333333333333333
d1 = Floor( d1 * p ) / p
d2 = 0.63333
// d1 = d2 because both are represented by 0.6333299999999999