Learning about doubles

Getting Started
1

Today I found that:

Dim i As Integer i = 5.1 * 100 // result 509
I was told that ‘such is the nature of doubles’, in this case 5.1 and 100 are converted to doubles and then casts back to an integer. There are workaround to make sure I get 510, like using Ceil or use single/currency for the operation and then assign to integer.

Then tested:

Dim i As Integer i = 5.1 * 10 // result 51 i = 5.1 * 1000 // result 5100
so the results are not consistent, I expected 50 and 5099. I guess the number of decimals after the operation change the value enough to alter the final integer/double value.

Then I tested some more and found that:

Dim i As Integer i = 5.1 * 10 * 10 // result 510 i = 10 * 10 * 5.1 // result 509
and

Dim i As Double i = 5.1 * 10 * 10 // 510. i = 10 * 10 * 5.1 // 509.9999999999999432

Looking at those results, I have to be careful with doubles.

2

When converting from any floating point type of number to an integer, the fractional part is truncated. To round the number to the nearest integer, simply add 0.5 to the number before the conversion.

Dim i As Integer
i = (5.1 * 100) + 0.5 // result 510
3

Doubles dont work like that

Remember operator precedence:

i = 5.1 * 10 * 10 // result 510
i = 10 * 10 * 5.1 // result 509

It is the same as:

i = 51 * 10 // result 510
i = 100 * 5.1 // result 509

I try to never use them

4

If you then start to use them as integers, certainly…

If you are working with values in the dollars and cents range (eg no more than 2 decimal places) then you should probably work with ‘inflated integers’ where your internal workings are integers 100 times larger than the actual doubles.
Divide by 100 to display

[code]Dim i As Integer
i = 51 * 1000 // result 51000

msgbox "display version : " + str (i \100,“0.00”)[/code] //510.00

5

If you want an integer result from doubles math, you should round the result.

i = round(100 * 5.1) // 510

6

Thank you all for your comments.

Now imagine that you are new to Xojo and programming. That you read the docs about integer and you think about this:

Dim i As Integer i = 5.1
what do you expect? i = 5 right? Even without executing the code.

Now, you are presented with the following:

Dim i As Integer i = 5.1 * 100
what do you expect (without executing the code)?

I could say that I expect that to be 500, why? because I may believe that because I ‘Dim i As Integer’, everything at the right of the equal sign will be treated as integer, so integer of 5.1 = 5 and integer of 100 = 100, 5 * 100 = 500
Or I could say that the language will use some numeric datatype that can handle decimals and then convert to integer, getting 510 as a result.
How should I know that the 5.1 and 100 will be converted to Doubles, and because the nature of the doubles 5.1 will be 5.099999… and i will end as 509 while

Dim i As Integer i = 5.11 * 100 // 511

Now that I know that this is happening, I can make better choices, like using Round, using 10000 then /100 or other option.

I guess what I’m trying to ask, is every operation treated as double and then cast back to the datatype in use? Or just in some cases?
Example:

Dim c As Currency c = 5.1 * 100

7

[quote]Dim i As Integer
i = 5.1 * 100
what do you expect (without executing the code)?[/quote]

510

Nobody would expect 509 without running the code
I can rationalise why it happens, but only in hindsight.

And it does expose the problem of double compares, which are always dangerous;

ie if 5.1 * 100 = 510 then

8

After reading the entries above, I was thinking:

Dim i As Integer i = (5.1 * 100)

so 5.1 * 100 will be evaluated, then the result stored into i.

No, it was not the case. For some reason, 5.1 is evaluated (resized to integer) and then * 100 and the result is stored into i.

The question is: is this the right way to do the things (I suppose so, else Xojo engineer(s) would wrote Xojo differently).

Conclusion: an additional entry in the LR is required (even if this is learned in the CSI studies).

A simple compare with how Encodings works with strings let me think…
a String without encoding is UTF8 by default. (Read String in the LR).

So, before making math on data, Xojo reads the datatype of the variable who will hold the value and work accordingly (set the “formula” to/as Integer and resolve it).

What as developers must we do ?
We have to know how Xojo (the development language) works and code accordingly to what we want: do we want an accurate result (with the decimal part) ?
Use the correct Datatype declared variable (a Double or an Integer or…).

Remember: Floor and Ceil are here for something. What if you are programming a Billing software (accounting software) ?

9

I was a little bit skeptical that something as mundane as “5.1” would cause such an issue. So of course I replicated the example in Xojo as saw the same 509 as everyone else did.

So as an experiment, I did it in Swift to see what IT did.

var i : Int = Int(5.1*100)
print("I=\\(i)")

basically the only difference is having to add the CAST, as SWIFT if very strict with datatypes… but that not withstanding… it still came out 509 … and by default Swift uses a 64bit Double

10

5.1 * 100 is a 509.9999999999999432 Double. Unhappily casting it to an integer does a Trunc() instead of a Round(), so, that’s why the 509. Use manually a Round() to achieve the expected result.

Dim i As Integer = round(5.1 * 100)

Dim d As Double = 5.1 * 100
Dim j As Integer = Round(d)

11

But, interestingly, what do you expect as a result for Round(-5.1*100)? That’s why I made my own “round()” for my Money Class.

12

Function RoundInt(d As Double) As Integer // Rounds the value to the upper nearest positive integer or lower nearest larger negative If d<B Then Return -Round(Abs(d)) Return Round(d) End Function

13

Thank you again.

So this means that every operation Xojo does is with doubles and then cast back to the corresponding datatype?

For example:

Dim c As Currency c = 5.1 * 100
5.1 * 100 is evaluated as double then when it is assigned to ‘c’ Xojo does some ‘fixing’/‘rounding’ so I get 510?

or with single:

Dim s As Single s = 5.1 * 100

14

The exact answer for this, only the compiler team (@Joe Ranieri ?) could answer precisely, but it would be a simpler way of doing things when the compiler noticed floating point arithmetic involved, promote all to Double, do the magic, cast the result back to the final form.

15

@Alberto De Poo — Or, instead of using a mixture of doubles, singles and integers in the same formula, you also can explicitly tell the compiler what you want. For example, you can use 100.0 instead of 100.

16

The compiler needs to automagically promote a 100 to a 100.0 when the expression contains floating parts.

17

It needs to obey some direct castings like:

Dim i As Integer = Integer(509.99)-9.
There’s a float, but as it’s final form would be Integer, the rest of the simple math should be int math, not floating math.

18

Xojo evaluates the data types on the right hand side of the equals sign (the data type of the receiving variable is not taken into consideration) and promotes all data to the type that most closely accommodates them all. Then it performs the operation.

dim i as integer

i = 5.1 * 100
integer = double * integer
integer = double * double
integer = double (truncated)

i = 5 * 100
integer = integer * integer
integer = integer (direct assignment)

As far as the original issue of getting a value of 509, it is not unique to Xojo. It is a result of how doubles are represented and is a problem in all programming languages. That is why you never represent monetary values as a double in an accounting application. You will be off by fractions of cents in your calculations.

On the other hand, doubles are completely precise for integer values and were used for holding integers greater than 32 bits before 64-bit variables became prevalent. A double gives you a precise 53-bit integer value.

19

If you’re interested in learning more about why some decimal numbers don’t have their exact floating point representation…
…then this is a good read: Bartosz Ciechanowski: Exposing Floating Point

20

Does Str(double) round the number, use Ceil or is doing something else?

Dim d As Double d = 5.1 * 100 // 509.9999999999999432 MsgBox Str(d) // 510