I have some fonts that contain characters in the range 127 - 135
By default, I gather that 127 onwards are treated as control characters of some kind.
When I try to draw them in Xojo, I just get blanks.
Yet symbols are defined.
How can I get these symbols to display on screen / in a listbox / in a PDF file when I embed the font?
Use DefineEncoding to set the string to the encoding that you want it to have, otherwise it will be UTF-8 by default. UTF-8 uses some of the values greater than 127 to indicate continuation bytes. You should look up UTF-8 in Wikipedia, the artcle there is quite helpful.
Defining or converting the encoding to UTF-8 is probably the best avenue, as indicated by @TimStreater .
For such cases where I am unsure of the existing encoding, and as a way to define encoding to UTF8 quickly, I added this method in a module. I did not note the original author, but if I recall correctly, the credit goes to @Greg_O . It is in a post somewhere on the Forum.
Public Function UTF8(extends s as String) As String
Select Case s.Encoding
Case Encodings.UTF8
// Don't modify if it's already UTF8
Return s
Case Nil
// Just set if it's not set
Return DefineEncoding(s, Encodings.UTF8)
Case Else
// otherwise, convert
Return ConvertEncoding(s, Encodings.UTF8)
End Select
the usage is most easy: mystring.UTF8 does the job.
Can you SHOW what are you talking about? There are many possibilities of interpretation going on here, like you reading some old content using DOS Code Pages, reading some unknown encodings, missing bits encoding as ASCII (7 bits) in intermediary steps, any other improper transformations during intermediary steps, etc.
In case UTF8 cannot be used and a single-byte encoding is required, I believe that ISOLatin1 (ISO 8859-1) is a single-byte encoding that would support characters above 126.
oups… I was really answering to @Jeff_Tullin
ASCII is a 7-bit code so anything beyond 127 isn’t ASCII but probably some 8-bit extension of the ASCII standard such as ISO Latin1. However there are dozens of 8-bit ASCII extensions so one has to find out which one it is. Once you’ve defined the encoding of your string with DefineEncoding all characters should be rendered correctly.
2 Likes
Not if preprocessed as ASCII in some way the bit 7 (8bit, 7 down to 0) was truncated.
Simple test:
In Paint Event of a Canvas:
g.DrawingColor = RGB(255,0,0)
'g.FontName = "My special Font Name"
g.FontSize = 24
g.DrawText Encodings.UTF8.Chr(128),20, 30
g.DrawText Encodings.UTF8.Chr(129),40, 30
g.DrawText Encodings.UTF8.Chr(130),60, 30
g.DrawText Encodings.UTF8.Chr(131),80, 30
g.DrawText Encodings.UTF8.Chr(132),100, 30
g.DrawText Encodings.UTF8.Chr(140),20, 60
g.DrawText Encodings.UTF8.Chr(141),40, 60
g.DrawText Encodings.UTF8.Chr(142),60, 60
g.DrawText Encodings.UTF8.Chr(143),80, 60
Paint nothing, but probably because there is othing to paint. But If I use 220 and following I get characters.
Xojo 2024r4.2
Sequoia 15.3.2
MacBook Pro m1
I do see characters above about 140 in the file.
DOSLatin1/2/3, IBMLatin1/2/3, UTF8, UTF16 , WindowsLatin1 don’t do it.
In this image, the symbols after 127 which I cannot get to appear, are shown with a red line through them.
While I appreciate that these characters ‘are supposed to not show up’, I know other apps can display them. I’m out of ideas. 
If anyone is interested, zip file with the font and a sample project is attached.
fonttest.zip (40.8 KB)
Under macOS I can inspect the font with Font Book and it shows the Unicode code points for all the glyphs in the font. It also reveals which standard character each glyph corresponds to:
.
Indeed the font appears to have issues but it appears to work (mostly):
I’ve found just six glyphs that don’t work as they don’t seem to have Unicode code points:
This set of glyphs doesn’t look like a regular code-page where the first one has byte value 0 and the last has byte value 255. Which is how most code pages are presented to us (the non UTF-8 ones, I mean). This is a collection of glyphs with Unicode values all over the place.
Almost certainly it was originally pooled from glyphs taken from other fonts.
How did you get to the big ones (which worked for you)?
Have-you tried to re-download the font from the seb site ?
PS: same table structure…
Same file.
I’ve copied those in Font Book (select the glyph and hit the Copy button) and pasted them in TextEdit. Some are easy to access directly; for example ‘W’ produces a black heart.
Seems some have huge character numbers 8000+
No wonder I cant find them around the 127-150 range.
I suspect the app I showed a screenshot from is doing some substitutions.