math

Getting Started
1

hello

I use all declared as Int32
when I try to pull the value using str () the result is always 0
I expect values ??of type
1.28606227 with str(wrr)
0.96085112 with str(hrr)

what am I doing wrong?

Thanks,
Rui

    tfb = tfocallenght * barlowr
    tfbr = 206.265 * pixelsize / tfb
    asw = width * tfbr / 60
    ash = height * tfbr / 60
    wr = width / binr / resizer
    hr = height / binr / resizer
    wrr = asw / 60
    hrr = ash / 60

  msgbox(str(wrr)) // check result
2

Hard to tell without seeing proper values there, but my guess is that your int’s, when divided by 60, are less than 1.

An int is a whole number.

3

integers will not do fractional math. You need to declare them as double.

Also, str() will round numbers so it’s better to use format() when outputting doubles.

4

Marc’s answer is better :slight_smile:

5

wonderful :smiley: works !!!

Thanks

Rui

6

Sometimes get values ??in this format
how can I get around this?

1.#INF00e
1.INF00e

and should show
2.85791617
2.13522472

Rui

7

Floating point math can result in special results like “infinite” “negative infinite” and “not a number”. This isn’t a Xojo bug but a limitation of floating point arithmetic.

8

Thanks Andrew.

9

[quote=28488:@Marc Zeedar]integers will not do fractional math. You need to declare them as double.

Also, str() will round numbers so it’s better to use format() when outputting doubles.[/quote]
Not 100% true. Str accepts a second parameter identical to Format. Format will respect international settings, Str will not.

10

Infinite and not-a-number results usually mean you have divided by zero.