Very strange casting error

General
1

Hi,

I experience an error which I cannot manage to solve. It’s probably a stupid thing but
I could not find the problem. This is the thing.
Look at the following lines. The first MsgBox gives 0.4, which is the correct
result. The second MsgBox, which is just 1-previous_number gives 1 instead
of 0.6…It’s not a thing related to MsgBox as also the variable storing the result
gives 1…which is the problem? :slight_smile:

Thanks!

MsgBox Str(6sum_d2/(n(n^2-1)))
MsgBox Str(1-6*(sum_d2/(n*(n^2-1))))

2

What happens if you rewrite the second line as:

MsgBox Str(1-(6sum_d2/(n(n^2-1))))

As you have it written your placement of the parens in the second line makes the equation operate in a different sequence

3

I mistyped in my original post:

MsgBox Str(6sum_d2/(n(n^2-1)))
MsgBox Str(1-6sum_d2/(n(n^2-1)))

anyway I also tried what you suggest but the result is the same… :frowning:

The strange thing is that only one particular calculation is giving me this error. Very very strange

4

Does the result change if you do 1.0 instead of 1:

MsgBox Str(1.0-6*sum_d2/(n*(n^2-1)))
5

[quote=224484:@Davide Pagano]
MsgBox Str(6sum_d2/(n(n^2-1)))
MsgBox Str(1-6*(sum_d2/(n*(n^2-1))))[/quote]

JUTS FWIW Str() is not a CAST - its a method in the framework which takes a numeric input & returns a string representation of that number

6

I’d make all the numeric representations doubles to avoid any confusion

MsgBox Str(1.0 - 6.0 * sum_d2 / (n * (n ^ 2.0 -1.0)))
7

Of course I know it…I mentioned “cast” in my OP to say that the result of the calculation seems a cast to integer (0.6 → 1) but I said it is not related to MsgBox as also the associated numerical variable “stores” that value.
As I could not figure out the issue I re-wrote the entire method from scratch using double as data type for all (which is also what Kem suggested). This solved the issue, but it is still a mystery to me…

8

What values of sum_d2 and n do you use to get 0.4? I’d like to try and see what happens.