Unexpected calculation results

[Dale_Arends]

I came across the following question on another programming forum and thought I’d try it on Xojo. Simply put, it said that

Dim resultd as Double
resultd = 24 * 60 * 60 * 180 * 1000

yielded an incorrect, negative result. In fact, it does unless at least one element is determinable as a double.

Dim resultd as Double
resultd = 24 * 60 * 60 * 180 * 1000.0

yields the correct result, 15552000000. So I decided to try other configurations.

Dim resultd As Double
Dim resultd1 As Double
Dim resulti As Integer
Dim resultiu As UInteger
Dim resultiu1 As UInt32
Dim resultiu2 As UInt64
resultd = 24 * 60 * 60 * 180 * 1000
resultd1 = 24 * 60 * 60 * 180 * 1000.0
resulti = 24 * 60 * 60 * 180 * 1000  
resultiu = 24 * 60 * 60 * 180 * 1000  
resultiu1 = 24 * 60 * 60 * 180 * 1000  
resultiu2 = 24 * 60 * 60 * 180 * 1000 

gave the following results

resultd : -1627869184
resultd1 : 15552000000
resulti : -1627869184
resultiu : 2664098112
resultiu1 : 2664098112
resultiu2 : 18446744072081682432

The resultd1 value, as a double with at least one element as a clear double value forcing floating point calculations, is correct. I understand, I think, that the problems were caused by bit overflow since the correct result needs more than 32 bits. What surprised me, with my limited understanding of math internals, was that the UInt64 result was so weird. I apparently incorrectly assumed that it would be in line with the floating point calculation result. Silly me.

[Bill_Gookin]

Interesting, in version 2020r2 on my Mac (10.15.17), it gives the answers I’d expect (15552000000 or 15552000000.00000, and resultiu1 still has the value 2667098112). But when I run it on Windows 10 (64 bit), I get the numbers you see.

I certainly wouldn’t expect different answers between Win and Mac.

[Greg_O_Lone]

Are you compiling as 32 or 64 bits?

[Ivan_Tellez]

@Dale_Arends

I’d try it on Xojo. Simply put, it said that

Extending a little your test:

Dim resultd As Double
Dim resultda As Double
Dim resulti32 As Int32
Dim resulti32a As Int32
Dim resultI64 As Int64
Dim resultI64a As Int64
Dim resultUI32 As UInt32
Dim resultUI32a As UInt32
Dim resultUI64 As UInt64
Dim resultUI64a As UInt64
Dim resultI As Integer
Dim resultIa As Integer


resultd = 24 * 60 * 60 * 180 * 1000
resultda = 24 * 60 * 60 * 180 * 1000.0
resulti32 = 24 * 60 * 60 * 180 * 1000
resulti32a = 24 * 60 * 60 * 180 * 1000.0
resultI64 = 24 * 60 * 60 * 180 * 1000
resultI64a = 24 * 60 * 60 * 180 * 1000.0
resultUI32 = 24 * 60 * 60 * 180 * 1000
resultUI32a = 24 * 60 * 60 * 180 * 1000.0
resultUI64 = 24 * 60 * 60 * 180 * 1000
resultUI64a = 24 * 60 * 60 * 180 * 1000.0
resultI = 24 * 60 * 60 * 180 * 1000
resultIa = 24 * 60 * 60 * 180 * 1000.0

@Greg_O_Lone

Are you compiling as 32 or 64 bits?

BUILT 32 bits

Double : -1627869184 WRONG
Double. : 15552000000 OK

Int32 : -1627869184 EXPECTED
Int32. : -1627869184 EXPECTED

Int64 : -1627869184 WRONG
Int64. : 15552000000 OK

UInt32 : 2667098112 EXPECTED
UInt32. : 2667098112 EXPECTED

UInt64 : 18446744072081682432 WRONG
UInt64. : 15552000000 OK

Integer : -1627869184 EXPECTED
Integer. : -1627869184 EXPECTED

Built 64 Bits

Double : 15552000000 OK
Double. : 15552000000 OK

Int32 : -1627869184 EXPECTED
Int32. : -1627869184 EXPECTED

Int64 : 15552000000 OK
Int64. : 15552000000 OK

UInt32 : 2667098112 EXPECTED
UInt32. : 2667098112 EXPECTED

UInt64 : 15552000000 OK
UInt64. : 15552000000 OK

Integer : 15552000000 OK
Integer. : 15552000000 OK

Looks like Xojo Ignores the data type in the left part and just uses Int32 to make the operations.

@Bill_Gookin

I certainly wouldn’t expect different answers between Win and Mac.

Kind of worryng that xojo gives different results depending on the OS

[Christian_Schmitz]

You are right. Xojo doesn’t look what is left of the assignment here.
It will look on the constants. If they fit in int32, they will be int32 and so calculation is done in int32.

[Bill_Gookin]

@Greg_O_Lone

Are you compiling as 32 or 64 bits?

Ah, I forgot that was still an option for Windows. Changing it to 64 bit makes it match. It’s a little unintuitive to look there even when the build checkbox isn’t checked.

[Ivan_Tellez]

@Ivan_Tellez

Looks like Xojo Ignores the data type in the left part and just uses Int32 to make the operations.

In further inspection looks like this is the normal compiler behavior. The compiler promotes values to that of the most precise type in the expression before doing a calculation without considering the Data Type where you are going to store the result.

So, another option to avoid doubles can be to use a Int64 in the expression:

Dim k As Int64 = 1000
Dim resultI64 As Int64
resultI64 = 24 * 60 * 60 * 180 * k

[Markus_Winter]

You guys are a bit late to the party:

Xojo Math badly broken: 32bit problems in a 64bit environment

For a forthcoming article in xDev I had to use the Mod function and came across an interesting problem: From the Xojo documentation: result = number1 Mod number2 The Mod operator divides number1 by number2 and returns the remainder as result. If either number1 or number2 is a floating-point type, it is first coerced to Int32. If either number1 or number2 is an Int64, then both values are promoted to Int64. What happens with result = doubleValue mod intValue? result and intValue are Intege…

Or Bob’s blog. Or Thomas’s blog. Or …

Was a huge commotion a few months back.

I went into more detail in my xDev article, and yes, also stated that it cannot be that you get different results on your Mac at home and your PC at work. Especially as you can use 64bit integers in 32bit Windows.

[Markus_Winter]

@Ivan_Tellez

The compiler promotes values to that of the most precise type in the expression

No.

[Markus_Winter]

P.S. some languages do it properly:

var resultD : Double =  24 * 60 * 60 * 180 * 1000
print(resultD)

Result:

15552000000.0

[Dale_Arends]

@Greg_O_Lone

Are you compiling as 32 or 64 bits?

Yep. In 32 bit things are as I found. Changing to 64 bit and all except the UInt32 are “correct”.

[brian_franco]

Not really unintuitive as if you are developing for Windows you have to build for 32 and 64 as both currently are still supported

[KarenA]

@brian_franco

Not really unintuitive

I would say since one is assigning the result to a double, it definitely would not be intuitive to many non-pros…

How many would think to do this:
Dim resultD as Double = CType( 24 * 60 * 60 * 180 * 1000, double)`

-Karen

[brian_franco]

That is a different topic between pro’s and non Pro’s

Sounds like an addition to the documentation would be justifiable ?

[KarenA]

@brian_franco

That is a different topic between pro’s and non Pro’s

I was being kind

I suspect a significantly non zero # of pros would not initially realize the issue until they saw the negative result.

-Karen

[brian_franco]

why suspect?
If we want Xojo to succeed we should point out what needs to be altered in the doc’s for everyone

[Christian_Schmitz]

It’s really unexpected for people.
Maybe a feedback case could be made to have xojo compiler promote integers to 64-bit in 32-bit to make it perform like on a 64-bit CPU?

[KarenA]

@brian_franco

If we want Xojo to succeed we should point out what needs to be altered in the doc’s for everyone

Documenting something unintuitive is very important, but doing so won’t make it intuitive … or remembered … unless bitten hard few times!

-Karen

[brian_franco]

Understand that past can hurt, been there but why stop trying to help Xojo become better?
I mean for newbies and experienced because we will always encounter situations no matter how experienced we are in any language or stack

-Brian

[KarenA]

@brian_franco

Understand that past can hurt, been there but why stop trying to help Xojo become better?

My point is that the compiler should be “fixed” not the docs
-karen