double variable math question

please consider the following little piece of code:

dim a,b as double
a = 3 * 1.2
b = 3.6
if a < b then
msgbox "a less than b "
end
if a = b then
msgbox "a equal b "
end if

the result is that a is minor than b !!!

if I declare both the variables as currency, they result equal…

please explain me this thing, this is against all my beliefs about double calculations

Giulio

The results you are seeing are due to the inacurracy of doubles. Stop in the debugger and you will see that a = 3.5999999999999996 and b = 3.6000000000000001. When you declare them as currency, they are rounded to 2 decimal places to become =.
bottom line: Always format doubles.

Welcome to the wonderful world of representation of doubles in binary. When you store 3.6, you are storing its closest representation, 3.6000000000000001. The calculation, however, yields 3.5999999999999996, which is less.

If you want to be sure they are essentially the same to n decimal places, use something like this. (I am comparing to three decimal places by using 1,000 as my multiplier.)

  dim a,b as double
  a = 3.0 * 1.2
  b = 3.6
  
  dim compareA, compareB as Int64
  compareA = Round( a * 1000.0 )
  compareB = Round( b * 1000.0 )
  if compareA < compareB then
    MsgBox "a less than b "
  elseif compareA = compareB then
    MsgBox "a equal b "
  end if

BTW, note that I put a decimal point after my numeric constants, like “3.0” and “1000.0”. This tells the compiler that these are doubles and not integers, preventing unnecessary conversions.

[quote=67884:@Kem Tekinay]
BTW, note that I put a decimal point after my numeric constants, like “3.” and “1000.”. This tells the compiler that these are doubles and not integers, preventing unnecessary conversions.[/quote]

I’d strongly encourage you to use 3.0 and 1000.0 as the fact the compiler allows 3. and 1000. is a parsing bug actually :stuck_out_tongue:

Kem, I’ve edited your post slightly as there needs to be a digit after the decimal place (i.e. 1.0 instead of 1.). The compiler currently allows both, but it’s a bug.

Is that right? I always thought that was just shorthand, one I use all the time.

If that bug is fixed, I sure hope a compiler error comes up so I can fix the places I’ve done that. Because I can tell you, I’ve done it a lot.

[quote=67893:@Kem Tekinay]Is that right? I always thought that was just shorthand, one I use all the time.

If that bug is fixed, I sure hope a compiler error comes up so I can fix the places I’ve done that. Because I can tell you, I’ve done it a lot.[/quote]

It will.

perhaps it is time someone wrote a DOUBLE tutorial and posted it, as this questionor a variation comes up every few weeks…
It would save the re-explanation, by pointing the poster to a standard explanation

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

I also trust there is a compelling reason (perhaps one you can share) to fix it at all? It’s quite handy.

At any rate, this RegEx will find most occurrences within a project while avoiding quoted numbers.

([^"]+\\b\\d+)\\.(?!\\d)

You can replace that with \\1.0. Numbers within comments will be flagged by this.

You can also use the Equals method to compare equality of doubles within a tolerance:

[code]Dim a, b As Double
a = 3.0 * 1.2
b = 3.6

If a.Equals(b, 2) Then
// “Equal”
End If[/code]

I hate when I don’t know stuff I should know…

Mostly the eventual desire to allow extension methods on literals (case 20830).

I don’t see the problem.

1000.Method (extension method on integer literal)
1000.0 (double literal)
1000.0.Method (extension method on double literal)
1000. (double literal)
1000…Method (syntax error)

No?

[quote=67922:@Kem Tekinay]I don’t see the problem.

1000.Method (extension method on integer literal)
1000.0 (double literal)
1000.0.Method (extension method on double literal)
1000. (double literal)
1000…Method (syntax error)

No?[/quote]

That’s what I thought initially too, but there’s an awesome fun edge case:

1.e4method

Is this three tokens ( REAL, ‘.’, IDENTIFIER ) or is it two tokens (REAL, IDENTIFIER)? And what about:

1.emethod

The compiler currently also treats ‘1.e’ as a Double literal, which is also going to become invalid.

You’re interested in 123.456.myFun but i++ is too much. I can’t figure you guys out :stuck_out_tongue:

Something like i+=1 might happen (and the other ones like -=, *=, /= )

i++ and/or ++i (with all the associated c semantics about pre & post increment) won’t - I think Joe’s even gone so far as to say WON’T quite definitively

[quote=68041:@Norman Palardy]Something like i+=1 might happen (and the other ones like -=, *=, /= )
[/quote]

might

You did read that part ?

Oh really?! Cool! :slight_smile: I must have misread those other threads. Thanks.