Compare doubles: 50000.3 > 50000.3 = true!?

General
1

I encounter some strange behaviour of a greater than comparison:

var teststring as string = "50000.3"
var testdouble as double = teststring.toDouble
var testcalc as double = 50000.1+0.2
var test as Boolean = testdouble > testcalc
System.DebugLog "test = "+test.ToString  // outputs true!?

why might that be?

2

Do you need to check if one is bigger or is it enough to know if they are equal / unequal?

https://documentation.xojo.com/api/code_execution/equals.html

1 Like
3

I would like to hear from Xojo, that the calculated 50000.3 is not smaller than the “string-originated” 50000.3
50000.3 > 50000.3 should be false, hence:
testdouble > testcalc should be false
But Xojo gives out a “true”!

4

check the debugger you’ll see a double is not to be expected as-is.

50000.3 could as well be stored as 50000.2999999999999999999999999999999999 in your computer.
It’s not a xojo thing, it’s how doubles work.

5 
System.DebugLog "testdouble = "+testdouble.toString+", testcalc = "+testcalc.ToString

gives out:
testdouble = 50000.300000, testcalc = 50000.300000

6

You’re not seeing the actual 15-digit fractional portion. ToString is rounding it. You cannot reliably compare double values. That’s what Equals is for: Equal to a given number of decimals. Or round both values yourself and compare. As @DerkJ said, it’s just the nature of double precision in computer programming. There’s a reason they’re not used for monetary applications.

7

FWIW,
50000.3 is stored as 50000.300000000003
50000.1 is 50000.099999999999
50000.1 + 0.2 is 50000.299999999999

So, yes, testdouble is greater than testcalc.

50000.300000000003 > 50000.299999999999

1 Like
8

Crazy. Thank you for the detailed explanation.

9

No. Try to express 0.1 in binary and you’ll understand …

11

Not crazy. It’s how floating point works.

1 Like
12

If the CPU could split the number (the integer and decimal parts) and use both as integers, this problem wouldn’t exist. But probably this would be incompatible with how they could or couldn’t work…

1 Like
13

Most pocket calculators work that way. But the deduction is incorrect. Try division or taking roots or sinus or numbers like pi etc and you are back to square one - you do Maths with limited precision.

14

Awesome, I didn’t know about Double.Equals, and I happen to have a need right now, thanks! This forum rocks :slight_smile:

5 Likes