Division Confusion

Why does this occur?

Dim n as Single = 48194
n = n / 10000
'n = 4.8193999999999999

I would expect n to be 4.8194. Same thing happens if n is a double.

Which version of Xojo?

2017r3 - Windows 10 - debugging in 32bit in the ide

And I find this interesting:

Dim n as single = 4.8194
'n = 4.8193999999999999

Basic computer math. For example it is not possible to express 0.1 in binary code, the best a computer can do is an approximation.

You CAN do it precisely but then you need to use a different math system. Have a look at Bob Delaney’s website, maybe he has something


While I understand the issue with Double precision…
I tested this exact example using both Single and Double datatypes
and the result I got out was exactly what went in 4.8194

also under macOS

With different results on MacOS versus Windows, what is the explanation? Is it really impossible to represent 4.8194 as a single or double on Windows? I have a work around by using Currency. However, if this is not a bug then I want to understand more about this issue so I can look out for it.

Single and Double both have precision issues depending on the exact value being represents
Google it… there are tons of discussions about this, including on this forum
The problem manifests itself mostly with DOUBLE, but I can see where singles might be affected as well
It is not a bug in Xojo, nor is it a bug in either macOS or Windows… .it is due to the IEEE numeric standards in place (Google that too)

What may be happening here, it that macOS uses a different method for implementing the standard and therefore may represent certain values more correctly than Windows might.


power of 2:

binary = 2 to the power of = decimal
10000000 = 2^8 = 256
01000000 = 2^7 = 128
01000000 = 2^6 = 64
00100000 = 2^5 = 32
00010000 = 2^4 = 16
00001000 = 2^3 = 8
00000100 = 2^2 = 4
00000010 = 2^1 = 2
00000001 = 2^0 = 1

so 3 = 2+1 = 00000011
and 5 is 4+1 = 00000101
12 is 8 + 4 = 00001100

You can extend this with negative power numbers to get fractions:
10000000 = 2^-1 = 0.5
01000000 = 2^-2 = 0.25
00100000 = 2^-3 = 0.125
00010000 = 2^-4 = 0.0625
00001000 = 2^-5 = 0.03125
00000100 = 2^-6 = 0.015625
00000010 = 2^-7 = 0.0078125
00000001 = 2^-8 = 0,00390625

so 4.8194 is 4 + 0,5 + 0.25 + 0.0625 + 0.00390625 + a rest
is 00000100 11 11010001 … and you have a rest of 0,00299375
(whole number + fraction)

There is no way to express 0.8194 in binary.

Trick: Multiply by 10,000 because 48194 CAN be expressed precisely in binary and divide the result again by 10,000. Same result, but you avoid rounding errors on the way. Just beware of overflow errors.

But Marcus, that does not explain why macOS represents it correctly and Windows does not

Because that is a rounded result. If the result is within a certain allowed deviation (around 15 decimal places??? Ask Norman.) then it is rounded. That’s why Xojo can tell you the result is 0.1 when it’s internal binary representation is not but it is “close enough” to be 0.1

Or in other words: if the result is 0.999 … (continues with nines) then presenting it as 1 is an error of 0.0001 (you added 0,0001), but presenting it as 0.999 is an error of 0.0009 (you dropped 0.0009) or nine times larger.

can you cite a source to support that?
as you are inferring that the macOS implemenation of IEEE goes beyond the published standard
Windows doesn’t implement the standard fully

and the issue can’t be with Xojo, but rather the internal implementation of doubles,
unless of course Xojo didn’t implement it the same way

Now THAT might be a bug.

I would suspect that they use some maths libraries (as Maths is a standard item) rather than write their own, maybe there is a difference between Mac and Windows implementations? Maybe there is a bug in how they implement them?

Not a clue, but a lot of that … :wink:

P.S. I remember at least three occasions in the last 15 years where the build in calculator on Mac and / or Windows had a bug and came up with wrong results … though one of those occasions (on Windows) was a bug in the math unit of the CPU …

The difference cannot be in the representation. It is instead a difference in the presentation. Xojo on MacOS apparently presents the rounded version, whereas the Windows version does not. Whether one or the other is a bug is a subject for debate. IEEE floating point representation (whether it be double or single) can only represent relatively few numbers exactly. Everything else is an approximation.

Bottom line, when you’re working with floating point numbers, always use Format() to display them, and never use them for financial applications.

IMHO if the Mac represents the number 4.8194 in the debugger exact like this and Windows shows it in the debugger as 4,8193999999999999 then that means that the Mac in the debugger shows the rounded value and Windows shows the closest representation of what realy is in the memory of the computer. Because Macs and most pc’s use the same Intel processors the differnce isn’t probably there.
As Marcus has perfectly showed in his calculation is that the exact value can not be presented in binary. In fact i think that if you could look in the memory of the Mac you would also see the same value as in the pc.
In the windows IDE we have seen this presentation more than once in the integer values for left, top, width and height, all showed as floats. It just a little difference between Mac and Windows IDE.