How to Compare UTF8 Strings visually equal

I have in a textarea the Korean characters "?? (1)
I have found 3 different binary compositions of this string.

EFBB BFEA B8B8 EC9D B420 2831 29 (length 7)
EAB8 B8EC 9DB4 2028 3129 (length 6)
EAB8 B8EC 9DB4 2020 2831 29 (length 7)

The first one is the string stored in memory (and also the only one on Windows)
The second one is when the text is read into a read only area.
The third one is after the text has entered the TextChange event.

Is there a way to account for the way these strings are in binary?
Is there a comparison method that I could be using.
Would it be different if I used Xojo.Core.Text

You need to do Unicode normalization.
We do have functions for that in MBS Plugins.

My confusion is this is what XOJO gives me. It should go from one space to another on the same machine instance and have the same binary values.

After a closer look.

  1. is with UTF-8 BOM on front. You can remove the first 3 bytes.
  2. is the right one.
  3. has two spaces instead of one.

So there is no problem, except maybe one on the chair. :wink:

The BOM addition makes a little sense.
That is easier to control for comparison, but I’m confused why it’s there in some cases and not in others.

I know what you mean. I added an improvement to my files, but I forgot to take the BOM off of the file. It bit back.