We ran into an issue with rounding. And it’s not because our rounding function, or even MBS’s rounding function doesn’t work, it’s because of the way XOJO is storing the value.
If you enter a value 32.175 in a text field or a variable, XOJO stores the value as 32.174999999. If you then try to round it two places, you ALWAYS get 32.17, which is incorrect.
Oddly, if you enter the value 32.165, and then round it to two placed, it round correctly to 32.17.
Anyone know a workaround and how many other numbers don’t round correctly because of the way xojo is storing these numbers?
An old trick is to add 0.005 (one more decimal place than you want), then multiply by 100 and round that, then divide by 100 to get your final answer.
dim d1 as double = 32.175
dim n as integer = round((d1 + .005) * 100)
dim d2 as double = n / 100
But this is why you never use doubles for apps that deal with money. It introduces creeping error when adding up monetary amounts. It is much better to use integer values and divide by 100 for display purposes.
Can 32.175 be represented exactly as a floating point number? If not, then its being represented as 32.174999999 is inherent in the hardware. Further, for it then to become 32.17 is not rounding, it’s truncation. How are you doing the rounding?
If you want to store non-whole numbers accurately you cannot use the Double (or Single) data type. This is a technical limitation of the data type in all languages and not a Xojo issue. There are various tricks which can help you avoid the limitation in some places but I have found from experience that you eventually hit a dead end.
The better solution is to use fixed point arithmetic.
You can roll your own using the Int64 data type, use the Xojo Currency data type or if you need something more complex you can try Bob Delaney’s fp plugin which is now being maintained by Björn Eiríksson at Einhuger: Einhugur Software - Open source projects
This is not a Xojo issue; it’s an issue related to the limited precision of floating point numbers. For example, in:
we see that:
“In computing, floating-point arithmetic is arithmetic that represents real numbers approximately, using an integer with a fixed precision, called the significand, scaled by an integer exponent of a fixed base.”
Forgive me for expecting rounding a floating point number to work correctly. You see, I came from Visual Foxpro and we are converting a financial application that used floating point numbers with 6 digits of precision and it handled floating point numbers correctly. Perhaps because they included their own rounding function that handles this without any problems.
I also wondered why our rounding function worked fine with 32.165 but did not work with 32.175? Of course, we only stumbled on this and in this particular case there was only 3 digits of precision, but in most cases we need 6 digits.
Also, correct me if I’m wrong, but since we need 6 digits of precision, the suggestion to use currency is a moot point since it only allows for 4 digits of precision.
Not trying to be cantankerous, just trying to find a solution for an issue that we need to resolve.
Well, I know nothing of any FoxPro but if it’s a financial application perhaps they have their own arithmetic library which handles numbers in software. Arithmetic done with Doubles will use the built-in floating point hardware, which is IEEE 754 compliant.
Nonetheless, several have explained to you why this is not a Xojo-only issue and offered potential solutions. As a programmer who has identified a problem, it’s now up to you to make it work.
If you still think that Xojo can make an improvement here, you should file an issue in tracker.xojo.com.
From Visual Foxpro’s command window I created two floating point numbers with 6 digits of precision and then used their internal ROUND function to round them to 2 decimals.
floats and doubles have this issue in all languages. There is no precise way to represent irrational numbers in floating point. This applies to IEEE floating point and to the older Microsoft floating point formats.
Because I write mostly business software, I avoid these issues in .NET using the Decimal type, which stores any number precisely without such issues, at the expense of a bit of memory and performance. Xojo lacks such a type; although there is a freeware plugin that implements a decimal type, I haven’t yet needed to use if for Xojo projects, so can’t vouch for it. Ominously, last I looked at it, the release notes mention fixing memory leaks (it’s written in C++) and the author “thinks” he got “all or most of them”. Not something you ever really want to read about a lib that will be frequently called.
