Xojo lost many UTF8 characters when using RTF file created outside of Xojo

General
#1

Using the code below, I keep diacritical vowels (, etc.), but lost curly quotes ():

[code] Dim RTF_TIS As TextInputStream

 RTF_TIS = TextInputStream.Open(RTF_FI) // RTF_FI is created far above (not displayed)
  RTF_TIS.Encoding = Encodings.UTF8
  TA.StyledText.RTFData = RTF_TIS.ReadAll()[/code]

The text was created on El Capitan / TextEdit.

The trouble appears also in another project :frowning:

In the RTF file, the curly quote character: is coded as \\'92.
Other non ASCII (encoded characters) appears correctly.

Idea(s) ?

#2

did you turn off smart quotes in System Prefererences?

#3

I do not think so. I check… and I do not know where to find what you said…

But: I can see in Apple’s TextEdit my curly quotes / ASCII $92(�) is on screen on MacOS… in Xojo ?

I will try to save the file from the TextArea, after adding them (so I will add code to export as rtf in the TA).

#4

The same text, saved from the TextArea as RTF have the curly quotes saved as \\u146 (remember ’ in the entry above).

Also:
Load the rtf file in Apple’s TextEdit shows the curly quotes, so I copy / pasted the data into the TextArea, then save it (from Xojo) to disk / reload on my Xojo application display the file correctly.

I have a strange and working solution (and I do not know why).

As usual, am I the only one ?

#5

System Prefs/Keyboard/Text:

uncheck “Use Smart Quotes for…”

#6

Thanks Tanner.

Dave: I have “Use Smart Quotes for…” Checked.

#7

Worst !

I downloaded LibreOffice minutes ago and recall this bug. (how is this related ?)

So, I loaded my correct rtf file (that file opens correctly in both Xojo and Apple’s TextEdit) in LibreOffice and exported it as rtf.

Then I run my project and call the help (fhis file) and… get two times the same diacritical vowels ! *

Was there people:
a. who actually use RTF (on Mac OS) ?
b. who beta tested rtf,
c. at Xojo, who tested rtf ?
this does not let me think there were some…

My production environment:
MacBook Pro 13"
El Capitan (all updates)
Xojo 2015r1 AND 2018r2
LibreOffice 6.1.0.3.

Of course, when I Copy / Paste the correct text and save it from withing the Xojo created application, there is no more problem. But what will say the client when he will do its own rtf files with his software ?

BTW: I do not checked the situation on Windows.

  • Of course, I get an eye on the rtf data.
#8

So your first attempt worked except for the curly quotes.

Try re-creating the same text using textedit - but have “Smart Quotes” disabled in the keyboard prefpane before you do. Now textedit should just use the regular ' character (ASCII 39) and not try to convert it to a “curly” quote.

Do this with a new textedit document, not copied and pasted from your original.

#9

Thanks for your answer Tanner, but… In my texts I use curly quotes, and others text, i replace ASCII quote with curly quotes.

Of course I know this will not be changed soon… so many things to do, so few time/people to achive them.

Also, I am able to replace at load time \e92 with \u146 (exact values to be checked) before applying the loaded text to the TextArea (TA).

At last, I know - because I searched why - why my vowels (éèàíìãñõ and so on) appears twice in TA Xojo when the rtf is generated with LibreOffice, and I can do the same kind of action I note above (replace the value with another / same kind of action).

But a bug is a bug. Who recall how buggy the Xojo rtf support was in the first release ?
(AFAIR), onlly ASCII text was supported.

I do not tested, but the standard French for " are « and »; are-they wrong too ? No they are OK.
(and the Germans „” ? They disappears…)

Actual work-around:
Create your text where you want *,
Fine tune it with the UTF8 characters conforms (mandatory) for your target country,
Copy the whole Text,
Paste it in a Xojo special application
Save the text from that application.

And with a little luck you will have a correctly displayed text.

My Windows 10 laptop is under repair, so I do not know how this behave under Windows.

  • You can even create it from Xojo, but you have to add a Search / Replace feature.

Edit:
Xojo generated rtf files default header is:

{\\rtf1\\ansi\\ansicpg1252

LibreOffice:
{\rtf1\ansi\deff3\adeflang1025

Apple’s TextEdit:
{\rtf1\ansi\ansicpg1252\cocoartf1404\cocoasubrtf470

#10

For some reason I was convinced RTF didn’t support Unicode, but only ansi and codepages. It seems from 1.5 it supports 16-bit unicode and the standard is from the 90s. I had headcanon for RTF (a format I otherwise adore) that was over almost 30 years outdated.

Man… I feel old.

#11

Eduardo:

(smile:) According to your post, you are > 30 yo, so, yes, you are old. :wink:

Unfortunately, this increase by one day, each day :frowning:

– Frank Sinatra first wife dies some weeks ago at 102. Time to go !

#12