Caret issues

[quote=107933:@jim mckay]I think this must be a syntax coloring thing… these two if statements should be the same length (I added enough spaces and single quotes at the beginning of the lines to get the start of the text to align…)

The caret is at the end of the second line… It looks like the caret position is correct, just the characters are being placed wrong (that cumulative rounding error)… maybe both could be calculated the same way? or the characters should be drawn based on the length of the unformatted text vs cumulative length of the formatted line?

This is with Palatino font… Menlo shows slight misalignment, Monaco looks correct…[/quote]

I suspect it’s an accumulating rounding error due to the StringWidth of each ‘fragment’ returning a non-integral value but being soared in an integer.

Edit: Seems Norman already said this :).

As a font designer, I have a pretty thorough knowledge of character spacing. What I meant is that I did not know the exact code used by Xojo for the editor canvas.

What I tried to explain afterwards in the post is why where to insert the text cursor with precision is inherently a difficult task when done character by character, because fonts internal measurement precision far exceeds the actual pixel by pixel rendition on screen. Hence the system reports fractional spacing when it is quantum in actuality.

[quote=107933:@jim mckay]I think this must be a syntax coloring thing… these two if statements should be the same length (I added enough spaces and single quotes at the beginning of the lines to get the start of the text to align…)

The caret is at the end of the second line… It looks like the caret position is correct, just the characters are being placed wrong (that cumulative rounding error)… maybe both could be calculated the same way? or the characters should be drawn based on the length of the unformatted text vs cumulative length of the formatted line?

This is with Palatino font… Menlo shows slight misalignment, Monaco looks correct…[/quote]

Proportional fonts certainly suffer from this more than fixed width fonts but they all do because OS X returns doubles, not integers, for text width & other text measurements.

Yes, but so does Linux and GDI+…

So, I’m thinking that since the IDE is written with Xojo, you’re using graphics.stringwidth which returns an integer, when you need the OS call that returns a double. I really wish that the graphics class could just use doubles in general… sigh. I’ve had to work around this sort of thing more times than I’d like to think about…

[quote=108169:@jim mckay]Yes, but so does Linux and GDI+…

So, I’m thinking that since the IDE is written with Xojo, you’re using graphics.stringwidth which returns an integer, when you need the OS call that returns a double. I really wish that the graphics class could just use doubles in general… sigh. I’ve had to work around this sort of thing more times than I’d like to think about…[/quote]

After the post I made about the differential between OpenType internal metrics precision and crude pixel rendition by the imager, I gave a bit of thought to designing a font specifically to avoid the rounding issue.

Using the simple example for StringWidth in the LR, I looked at the width of “N” and this gives 8.865234 which on a normal screen shows very well the gap between pixels and character width. The imager can generate only bitmaps, so the character can be only 8 or 9 pixels, but never fractional. This cumulated difference explains very well the caret slide. For that particular character, we are talking 1.18 per cent per character, so in theory it takes a little less than 100 characters to have a cumulative error of one character width. To make things more difficult to manage, as each character in a proportional font may have a different delta between imaging and internal metrics, it becomes very difficult to calculate and compensate, if at all possible.

… / … A while later…

I have worked on the font issue. It is indeed possible to modify a font internal metrics to approach values in pixels. With a monospaced font, I have been able to generate characters with a stringwidth of 10.998 at size 13, which is extremely close to 11 pixels. Now it seems the imager does not really get enough precision to bridge the 2 per then thousand gap, as I tried to compensate with the font internal metrics. But it is extremely close. So such a font should not accumulate more than 2 per ten thousand rounding error per character. It would require a very, very long line to have a one character width error.

Pass that proof of concept, I do not plan on producing any proportional editor special font in the near future : I do not have time to painfully work on each character in the font to make sure it fits between pixels.

I am simply happy to have been able to reconcile the complexity of going from vector based OpenType (or TrueType) to pixel based screen and fractional string width measurement :slight_smile:

I’m thinking the answer lies in rendering each style run at the position it would be in a string without styles. Bold and italic are problematic though as they can change the width of the glyph vs unstyled text, but can be ignored without too bad an effect. If the textSize changes run to run, forget it…

Works great for color though. Not perfect by any means, but if anyone needs a workaround for the lack of double precision when drawing multicolored text…
For example…

[code] dim xCumulative as integer
if sText=nil then Return

for i as integer=0 to sText.StyleRunCount-1
dim r As StyleRun=sText.StyleRun(i)

g.Bold=false
g.Italic=False
// calculate x position ignoring bold and italic...
dim xCalc As integer=g.StringWidth(sText.Text.Left(sText.StyleRunRange(i).StartPos))

g.ForeColor=r.TextColor
g.TextFont=r.Font
g.Bold=r.Bold
g.Italic=r.Italic
g.Underline=r.Underline
g.TextSize=r.Size

//draw based on the width of the string up to here..
g.DrawString(r.Text, xCalc, g.TextHeight+5)

//draw based on the cumulative width of each style run up to here
g.DrawString(r.Text,xCumulative,(g.TextHeight+5)*2)
xCumulative=xCumulative+g.StringWidth(r.Text)

next

//draw with no style
g.DrawString(sText.Text,0,(g.TextHeight+5)*3)[/code]

[quote=108169:@jim mckay]
So, I’m thinking that since the IDE is written with Xojo, you’re using graphics.stringwidth which returns an integer[/quote]
Hasn’t returned an integer since 2007
http://documentation.xojo.com/index.php/Graphics.StringWidth

I’ll be darned. Learn something new everyday (or every 7 years I suppose)

[quote=108289:@Norman Palardy]Hasn’t returned an integer since 2007
http://documentation.xojo.com/index.php/Graphics.StringWidth[/quote]

And that is where the problem lies. On the screen positions are integers when the system uses double. At 72dpi one pixel = 1 point. With Retina at 144dpi precision only doubles so it should be possible to have a 1.5 pixel position, but yet in the case of the character “N” above that actually measures 9 pixels, StringWidth reports 8.865234 when it should be 9. I guess in that case rounding is fairly easy, but what if StringWidth where 8.5 ? Has the imager chosen to display 8 or 9 pixels ?

There is no escaping the fact that apparently StringWidth is based on the OpenType font internal metrics which has between 800 to 1000 ems (or “points”) per average character in width, when the actual display has 8-10 pixels. In the case of an average TrueType it is even worse, as the internal metrics use up to 2048 ems.

To get back to the original problem of the caret being off position, though, in all fairness on long lines the issue is not always that terrible, and seems to appear somewhat at random. I was unable to reproduce the issue by simply cutting and pasting text, though I did encounter it from time to time in real coding situations. Xojo has done a good job compensating the rounding error for day to day operation. Now I wonder why the same issue does not appear in TextAreas. Has Apple used some sort of trick from its sleeve to prevent the caret shift ?

My original thought was that if the position accumulator were a double, then the on-screen positions would only be off by, at most, a half pixel, and the caret would be off by the same… then I started thinking stringwidth still returned an integer…
So back to 2014… can’t you just use a double to accumulate and not round until the actual drawing takes place? ( using the word ‘just’ very loosely here… I know there’s got to be a lot more to it :slight_smile: )

I suspect the fractional character width accumulation may not quite reflect the actual pixel rendition. Your example here :

shows a strange dimensional shift between the exact two strings on the screen, which implies some pixels have been added in the second.

Looking closely, this shows a white pixel has been added in front of the second ‘and’ after the light blue ‘1’ :

Now the question is, where does this white pixel comes from ? Is it the result of random rounding error on the part of the font imager, or is it the result of character by character positioning done by the Xojo code rounded from the fractional character spacing ?

A drawstring with the System font at point size 13 (which I experimented to be the same as zero) should give exactly the same rendition no matter the color chosen, yet the blue line has enough pixels added in the spaces that at the end of the line the caret is shifted one character position. This is odd. System imaging or Xojo code, why does the change in color implies the wrong character spacing in the blue line ?

I strongly suspect some rounding error in the Xojo line display code, since colors are applied to parts of the string that do not in the brown one. It would make sense to draw each color one by one, so the wrong position of the ‘and’ after the ‘1’ would be the result of an inadequate calculus of the ‘1’ width.

To head off some of the speculation here…

As Norman has stated before, I believe that the problem is quite literally that each run (i.e. each separately colored segment) is storing its width as an integer instead of a double. The StringWidth of the run will return a double, get stored into that integer, and resulting in more and more rounding ‘errors’. This bug can be made especially visible on a line with a lot of runs.

[quote=108367:@Joe Ranieri]To head off some of the speculation here…

As Norman has stated before, I believe that the problem is quite literally that each run (i.e. each separately colored segment) is storing its width as an integer instead of a double. The StringWidth of the run will return a double, get stored into that integer, and resulting in more and more rounding ‘errors’. This bug can be made especially visible on a line with a lot of runs.[/quote]

So in the case of the '1’followed by space above, the value stored exceeds the actual pixel width. Nitty gritty inside the line display. Thank you for clarifying.

I just checked if the special monospace font I devised was able to alleviate the problem with Jim’s example, and it does. Both lines are exactly identical, no more extra or missing pixels. If I get some time I will try to create a version more suitable for everyday use.