# double variable math question

**URL:** <https://forum.xojo.com/t/double-variable-math-question/14619>\
**Category:** General\
**Created:** [February 26, 2014, 4:05pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619 "2014-02-26T16:05:33Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Giulio\_Mastrosanti](https://forum.xojo.com/user_avatar/forum.xojo.com/giulio_mastrosanti/32/2174_2.png) [@Giulio\_Mastrosanti](https://forum.xojo.com/u/Giulio_Mastrosanti)\
**Post date:** [February 26, 2014, 4:05pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/1 "2014-02-26T16:05:33Z")

</div>

please consider the following little piece of code:

dim a,b as double  
a = 3 \* 1.2  
b = 3.6  
if a \< b then  
msgbox "a less than b "  
end  
if a = b then  
msgbox "a equal b "  
end if

the result is that a is minor than b !!!

if I declare both the variables as currency, they result equal…

please explain me this thing, this is against all my beliefs about double calculations

Giulio

---

<div class="post-metadata">

**Author:** ![Roger\_Clary](https://forum.xojo.com/letter_avatar_proxy/v4/letter/r/eada6e/32.png) [@Roger\_Clary](https://forum.xojo.com/u/Roger_Clary)\
**Post date:** [February 26, 2014, 4:19pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/2 "2014-02-26T16:19:00Z")

</div>

The results you are seeing are due to the inacurracy of doubles. Stop in the debugger and you will see that a = 3.5999999999999996 and b = 3.6000000000000001. When you declare them as currency, they are rounded to 2 decimal places to become =.  
bottom line: Always format doubles.

---

<div class="post-metadata">

**Author:** ![Kem\_Tekinay](https://forum.xojo.com/user_avatar/forum.xojo.com/kem_tekinay/32/154_2.png) [@Kem\_Tekinay](https://forum.xojo.com/u/Kem_Tekinay)\
**Post date:** [February 26, 2014, 4:19pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/3 "2014-02-26T16:19:46Z")

</div>

Welcome to the wonderful world of representation of doubles in binary. When you store 3.6, you are storing its closest representation, 3.6000000000000001. The calculation, however, yields 3.5999999999999996, which is less.

If you want to be sure they are essentially the same to n decimal places, use something like this. (I am comparing to three decimal places by using 1,000 as my multiplier.)

```auto
  dim a,b as double
  a = 3.0 * 1.2
  b = 3.6
  
  dim compareA, compareB as Int64
  compareA = Round( a * 1000.0 )
  compareB = Round( b * 1000.0 )
  if compareA < compareB then
    MsgBox "a less than b "
  elseif compareA = compareB then
    MsgBox "a equal b "
  end if
```

BTW, note that I put a decimal point after my numeric constants, like “3.0” and “1000.0”. This tells the compiler that these are doubles and not integers, preventing unnecessary conversions.

---

<div class="post-metadata">

**Author:** ![Norman\_P](https://forum.xojo.com/letter_avatar_proxy/v4/letter/n/858c86/32.png) [@Norman\_P](https://forum.xojo.com/u/Norman_P)\
**Post date:** [February 26, 2014, 4:50pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/4 "2014-02-26T16:50:53Z")

</div>

[quote=67884:@Kem Tekinay]  
BTW, note that I put a decimal point after my numeric constants, like “3.” and “1000.”. This tells the compiler that these are doubles and not integers, preventing unnecessary conversions.[/quote]

I’d strongly encourage you to use 3.0 and 1000.0 as the fact the compiler allows 3. and 1000. is a parsing bug actually 😛

---

<div class="post-metadata">

**Author:** ![Joe\_Ranieri](https://forum.xojo.com/letter_avatar_proxy/v4/letter/j/43a26b/32.png) [@Joe\_Ranieri](https://forum.xojo.com/u/Joe_Ranieri)\
**Post date:** [February 26, 2014, 4:51pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/5 "2014-02-26T16:51:32Z")

</div>

Kem, I’ve edited your post slightly as there needs to be a digit after the decimal place (i.e. `1.0` instead of `1.`). The compiler currently allows both, but it’s a bug.

---

<div class="post-metadata">

**Author:** ![Kem\_Tekinay](https://forum.xojo.com/user_avatar/forum.xojo.com/kem_tekinay/32/154_2.png) [@Kem\_Tekinay](https://forum.xojo.com/u/Kem_Tekinay)\
**Post date:** [February 26, 2014, 4:54pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/6 "2014-02-26T16:54:51Z")

</div>

Is that right? I always thought that was just shorthand, one I use all the time.

If that bug is fixed, I sure hope a compiler error comes up so I can fix the places I’ve done that. Because I can tell you, I’ve done it a lot.

---

<div class="post-metadata">

**Author:** ![Joe\_Ranieri](https://forum.xojo.com/letter_avatar_proxy/v4/letter/j/43a26b/32.png) [@Joe\_Ranieri](https://forum.xojo.com/u/Joe_Ranieri)\
**Post date:** [February 26, 2014, 4:55pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/7 "2014-02-26T16:55:29Z")

</div>

[quote=67893:@Kem Tekinay]Is that right? I always thought that was just shorthand, one I use all the time.

If that bug is fixed, I sure hope a compiler error comes up so I can fix the places I’ve done that. Because I can tell you, I’ve done it a lot.[/quote]

It will.

---

<div class="post-metadata">

**Author:** ![DaveS](https://forum.xojo.com/letter_avatar_proxy/v4/letter/d/848f3c/32.png) [@DaveS](https://forum.xojo.com/u/DaveS)\
**Post date:** [February 26, 2014, 4:56pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/8 "2014-02-26T16:56:39Z")

</div>

perhaps it is time someone wrote a DOUBLE tutorial and posted it, as this questionor a variation comes up every few weeks…  
It would save the re-explanation, by pointing the poster to a standard explanation

---

<div class="post-metadata">

**Author:** ![Norman\_P](https://forum.xojo.com/letter_avatar_proxy/v4/letter/n/858c86/32.png) [@Norman\_P](https://forum.xojo.com/u/Norman_P)\
**Post date:** [February 26, 2014, 5:09pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/9 "2014-02-26T17:09:36Z")

</div>

[http://en.wikipedia.org/wiki/Floating\_point#Accuracy\_problems](http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems)

---

<div class="post-metadata">

**Author:** ![Kem\_Tekinay](https://forum.xojo.com/user_avatar/forum.xojo.com/kem_tekinay/32/154_2.png) [@Kem\_Tekinay](https://forum.xojo.com/u/Kem_Tekinay)\
**Post date:** [February 26, 2014, 5:18pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/10 "2014-02-26T17:18:47Z")

</div>

I also trust there is a compelling reason (perhaps one you can share) to fix it at all? It’s quite handy.

At any rate, this RegEx will find most occurrences within a project while avoiding quoted numbers.

```auto
([^"]+\\b\\d+)\\.(?!\\d)
```

You can replace that with `\\1.0`. Numbers within comments will be flagged by this.

---

<div class="post-metadata">

**Author:** ![Paul\_Lefebvre](https://forum.xojo.com/user_avatar/forum.xojo.com/paul_lefebvre/32/3152_2.png) [@Paul\_Lefebvre](https://forum.xojo.com/u/Paul_Lefebvre)\
**Post date:** [February 26, 2014, 5:23pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/11 "2014-02-26T17:23:36Z")

</div>

You can also use the [Equals](http://documentation.xojo.com/index.php/Equals) method to compare equality of doubles within a tolerance:

[code]Dim a, b As Double  
a = 3.0 \* 1.2  
b = 3.6

If a.Equals(b, 2) Then  
// “Equal”  
End If[/code]

---

<div class="post-metadata">

**Author:** ![Kem\_Tekinay](https://forum.xojo.com/user_avatar/forum.xojo.com/kem_tekinay/32/154_2.png) [@Kem\_Tekinay](https://forum.xojo.com/u/Kem_Tekinay)\
**Post date:** [February 26, 2014, 5:26pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/12 "2014-02-26T17:26:31Z")

</div>

I hate when I don’t know stuff I should know…

---

<div class="post-metadata">

**Author:** ![Joe\_Ranieri](https://forum.xojo.com/letter_avatar_proxy/v4/letter/j/43a26b/32.png) [@Joe\_Ranieri](https://forum.xojo.com/u/Joe_Ranieri)\
**Post date:** [February 26, 2014, 5:41pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/13 "2014-02-26T17:41:21Z")

</div>

> [@67904:@Kem Tekinay](#):
>
> I also trust there is a compelling reason (perhaps one you can share) to fix it at all? It’s quite handy.

Mostly the eventual desire to allow extension methods on literals (case 20830).

---

<div class="post-metadata">

**Author:** ![Kem\_Tekinay](https://forum.xojo.com/user_avatar/forum.xojo.com/kem_tekinay/32/154_2.png) [@Kem\_Tekinay](https://forum.xojo.com/u/Kem_Tekinay)\
**Post date:** [February 26, 2014, 6:16pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/14 "2014-02-26T18:16:24Z")

</div>

I don’t see the problem.

1000.Method (extension method on integer literal)  
1000.0 (double literal)  
1000.0.Method (extension method on double literal)  
1000. (double literal)  
1000…Method (syntax error)

No?

---

<div class="post-metadata">

**Author:** ![Joe\_Ranieri](https://forum.xojo.com/letter_avatar_proxy/v4/letter/j/43a26b/32.png) [@Joe\_Ranieri](https://forum.xojo.com/u/Joe_Ranieri)\
**Post date:** [February 26, 2014, 6:21pm UTC](https://forum.xojo.com/t/double-variable-math-question/14619/15 "2014-02-26T18:21:29Z")

</div>

[quote=67922:@Kem Tekinay]I don’t see the problem.

1000.Method (extension method on integer literal)  
1000.0 (double literal)  
1000.0.Method (extension method on double literal)  
1000. (double literal)  
1000…Method (syntax error)

No?[/quote]

That’s what I thought initially too, but there’s an awesome fun edge case:

```auto
1.e4method
```

Is this three tokens ( REAL, ‘.’, IDENTIFIER ) or is it two tokens (REAL, IDENTIFIER)? And what about:

```auto
1.emethod
```

The compiler currently also treats ‘1.e’ as a Double literal, which is also going to become invalid.

---

<div class="post-metadata">

**Author:** ![Will\_Shank](https://forum.xojo.com/letter_avatar_proxy/v4/letter/w/f475e1/32.png) [@Will\_Shank](https://forum.xojo.com/u/Will_Shank)\
**Post date:** [February 27, 2014, 1:22am UTC](https://forum.xojo.com/t/double-variable-math-question/14619/16 "2014-02-27T01:22:17Z")

</div>

> [@67908:@Joe Ranieri](#):
>
> Mostly the eventual desire to allow extension methods on literals (case 20830).

You’re interested in 123.456.myFun but i++ is too much. I can’t figure you guys out 😛

---

<div class="post-metadata">

**Author:** ![Norman\_P](https://forum.xojo.com/letter_avatar_proxy/v4/letter/n/858c86/32.png) [@Norman\_P](https://forum.xojo.com/u/Norman_P)\
**Post date:** [February 27, 2014, 3:28am UTC](https://forum.xojo.com/t/double-variable-math-question/14619/17 "2014-02-27T03:28:15Z")

</div>

> [@68025:@Will Shank](#):
>
> You’re interested in 123.456.myFun but i++ is too much. I can’t figure you guys out 😛

Something like i+=1 _might_ happen (and the other ones like -=, \*=, /= )

i++ and/or ++i (with all the associated c semantics about pre & post increment) won’t - I think Joe’s even gone so far as to say WON’T quite definitively

---

<div class="post-metadata">

**Author:** ![KarenA](https://forum.xojo.com/letter_avatar_proxy/v4/letter/k/c89c15/32.png) [@KarenA](https://forum.xojo.com/u/KarenA)\
**Post date:** [February 27, 2014, 3:34am UTC](https://forum.xojo.com/t/double-variable-math-question/14619/18 "2014-02-27T03:34:59Z")

</div>

[quote=68041:@Norman Palardy]Something like i+=1 _might_ happen (and the other ones like -=, \*=, /= )  
[/quote]

---

<div class="post-metadata">

**Author:** ![Norman\_P](https://forum.xojo.com/letter_avatar_proxy/v4/letter/n/858c86/32.png) [@Norman\_P](https://forum.xojo.com/u/Norman_P)\
**Post date:** [February 27, 2014, 3:39am UTC](https://forum.xojo.com/t/double-variable-math-question/14619/19 "2014-02-27T03:39:28Z")

</div>

> [@68044:@Karen Atkocius](#):
>

_might_

You did read that part ?

---

<div class="post-metadata">

**Author:** ![Will\_Shank](https://forum.xojo.com/letter_avatar_proxy/v4/letter/w/f475e1/32.png) [@Will\_Shank](https://forum.xojo.com/u/Will_Shank)\
**Post date:** [February 27, 2014, 4:09am UTC](https://forum.xojo.com/t/double-variable-math-question/14619/20 "2014-02-27T04:09:17Z")

</div>

> [@68041:@Norman Palardy](#):
>
> Something like i+=1 _might_ happen

Oh really?! Cool! 🙂 I must have misread those other threads. Thanks.

[Next page](https://forum.xojo.com/t/double-variable-math-question/14619.md?page=2)
