# Any Idea Why I Get Values With Decimals When Passing In Integers?

``````Function PointDistance(p1 as Point, p2 as Point) As double
dim x1, y1, x2, y2 as double
dim dx, dy as double

dx = p2.x - p1.x
dy = p2.y - p1.y

return Sqrt(dx * dx + dy * dy)
End Function``````

I am having problems with this method which is designed to calculate the distance between two points. The problem here appears to be that the value returned actually returns a value with a decimal point, rather than a whole number. But am I wrong to think that if both points do not have decimal points, the distance returned should also not have a decimal point.

I created a ‘Point’ class which uses variables of double type to store X and Y coordinates. I want the function to be able to use double values to compare distance but I also want this to get the correct value for whole numbers. Please let me know if I got something mixed up here and that the method does actually return the correct value by what I described.

An example of what happens is this:

If p1 coordinates are (25, 25) and p2 coordinates are (20, 45) then DX is -5 and DY is 20. But ‘Sqrt(dx * dx + dy * dy)’ returns 20.615528128088304.

Thanks

Because you’re using doubles for the calculations and return values. If you want integers only, use Int64s. Keep in mind that as integers, the value is not as accurate.

``Function PointDistance(p1 as Point, p2 as Point) As Integer  // or Int64 etc``

bearing in mind that your answer will by definition be off by a delta of >=0<1

the distance from (25,25) to (20,45) IS 20.6155… not 20

so depending on what you are using the result for , you might want to take the CEIL value not the default FLOOR value

[quote=258082:@Dave S]``` Function PointDistance(p1 as Point, p2 as Point) As Integer // or Int64 etc ```

bearing in mind that your answer will by definition be off by a delta of >=0<1

the distance from (25,25) to (20,45) IS 20.6155… not 20

so depending on what you are using the result for , you might want to take the CEIL value not the default FLOOR value[/quote]
Thanks. This sounds helpful for my functions. I will design a boolean parameter to tell the function to ceil the value before returning it.

[quote=258082:@Dave S]``` Function PointDistance(p1 as Point, p2 as Point) As Integer // or Int64 etc ```

bearing in mind that your answer will by definition be off by a delta of >=0<1

the distance from (25,25) to (20,45) IS 20.6155… not 20

so depending on what you are using the result for , you might want to take the CEIL value not the default FLOOR value[/quote]
Is using CEIL what you recommend for using distance for working with pixels?

Thanks

No, use Round().

the selection of ROUND, CEIL or FLOOR depends on why you are using an INTEGER value for a distance… each of them will affect the outcome in different ways as opposed to the true DOUBLE value

Problem with pixels is by definition there is no fractional value. I believe Round will probably do a better job. However, especially at low resolution, there will be lots of edge cases that will render strangely to the eye.

The best common example is screen fonts. They are using vectors that the imager transforms into pixels. At size 12 on a 72 dpi an average character is drawn on 8 x 10 pixels. System fonts are optimized to render in such environment, but other fonts will generally show strange obliques and curves.

The same problem occurs in automatic scaling down, for instance the app icon in Xojo. I have been regularly redrawing the 24x24 and 16x16 icons instead of counting on the IDE.

[quote=258121:@Michel Bujardet]Problem with pixels is by definition there is no fractional value. I believe Round will probably do a better job. However, especially at low resolution, there will be lots of edge cases that will render strangely to the eye.

The best common example is screen fonts. They are using vectors that the imager transforms into pixels. At size 12 on a 72 dpi an average character is drawn on 8 x 10 pixels. System fonts are optimized to render in such environment, but other fonts will generally show strange obliques and curves.

The same problem occurs in automatic scaling down, for instance the app icon in Xojo. I have been regularly redrawing the 24x24 and 16x16 icons instead of counting on the IDE.[/quote]
So, would this mean I should avoid using a function like this to get accurate calculations (for working with pixels)?

Thanks

Oliver, the definition of “accurate” is up to you, depending on what you need/want.

The algoritm for distance is by definition a floating point result… so if you need distance, then you need that equation, but its up to you to decide how to hand the answer. There is no “accurate” integer distance formula, just gross approximations

The same goes for calculating points on a arc (oval, circle) and almost any other geometric alogrithm (even determining intercept points on a straight line).

So you, as a developer need to decide for you application how best to map the floating point “real” data into you applications model

just want until you need to do interpolations or extrapolations… [quote=258200:@Dave S]Oliver, the definition of “accurate” is up to you, depending on what you need/want.

The algoritm for distance is by definition a floating point result… so if you need distance, then you need that equation, but its up to you to decide how to hand the answer. There is no “accurate” integer distance formula, just gross approximations

The same goes for calculating points on a arc (oval, circle) and almost any other geometric alogrithm (even determining intercept points on a straight line).

So you, as a developer need to decide for you application how best to map the floating point “real” data into you applications model

just want until you need to do interpolations or extrapolations… :)[/quote]
This cleared up the confusion. Thanks

You have no choice anyway, but you should be aware that some edge cases may present.

For instance, when you have a result of exactly 2.5. Round will give you 3, but what if 2 is better suited for whatever intent you have ?

As I said, the more pixels you have the less of an issue it is.

What I’m not clear on is why there being pixels has anything to do with having an integer distance. Maybe if you say why distance has to be an integer, what bearing it has on some pixel calculation, then someone might spot a better way to calculate it.

If you calculate the point of something on the screen. When you draw it to the screen.

I’d always do the calculation in the best precision you can and draw using those values
And let the underlying OS API’s etc handle turning your floating point values into integer pixel coordinates

It’s still not clear what distance calculation has to do with drawing. What are you trying to achieve. It sounds like you may be doing this the “hard way”.

You make a good point there. Thanks

Hi Greg,
Can you help me understand this?
Are you saying that because the OP is using a square root function, and his “answer” is not really an integer, that an Int64r is less accurate? Or that Int64 is somehow inherently less accurate? I thought that Double precision numbers were enormously big, but only really correct to some limited number of decimal places, whereas a large integer should be exact at whatever value.
Don

The result of Sqrt will have a fractional value. That fractional part is lost when you convert to integer, even Int64. Thus, the integer/int64 is less accurate (missing the fractional value).

Been reading this thread but I am still not sure what is the requirement. For me I will take the double (with decimals) as the answer I want. The actual screen pixel (?) distance is not important. The values of p1 and p2 should be stored somewhere is some arrays. The calculation results in double is the correct result and if needed also stored in a variable.

Plotting/drawing on the screen is only a very close approximate. Like Norman has said, let the OS do the conversion work.

The attached screen shot as example. Although this is not done in Xojo (yet), the whole routine will be the same. There are 3 “subgraphs”, each having its own yMax and yMin but sharing the same xMax and xMin. The drawing routine does not even keep track where the bars/lines are being drawn but the values of the bars/lines are retrieved from arrays when needed where-ever the mouse is clicked - the yellow price box is example.

I draw these on the screen (PictureBox - equivalent to Canvas in Xojo) using doubles and they get drawn where the OS wants it.

But this is my requirement. Don’t know what is yours. 