Encoding Question - instr FFEE

General
1

I need to match a string coming from a external file.

if data.instr(“Mobile”) > 0 then …

The string looks like this: FFFE 4D00 6F00 6200 6900 6C00 6500
…M.o.b.i.l.e.

instr isn’t working, I know it’s related to encoding, but haven’t worked out the solution. How do I convert …M.o.b.i.l.e to “Mobile” so the instr works?

2

The first two bytes are the giveaway. They represent the byte order marker for UTF-16LE. Strip the first two bytes, then define the encoding of the string as UTF-16LE, and it should start working for you.

https://en.m.wikipedia.org/wiki/Byte_order_mark

3

files that start with FFFE indicate UTF-16LE

OH… .Kem got here first. :frowning:

I knew it was important… I didn’t remember which encoding it defined… I have a routine that can usually “guess” a files encoding based on its contents… actually it was written by Joe Strout and updates recently by Thomas Tempelmann

4

It’s obvious, I don’t know what I’m doing…

dim mb as memory block            
dim data, newData as string
           data = tis.readall
            mb = data  // ??**	 It's what you Todo....
            if ( ( mb.byte(0)=&hFF ) and ( mb.byte(1)=&hFE ) ) then
              newData = mb.RightB(mb.size-2)
            end if
            data = ConvertEncoding(newData,  Encodings.UTF16LE)
            if data.InStr("Todo") > 0 then << Not working
            break
           end if
5

Looks good except you want to define the encoding, not convert it.

6

To clarify, use ConvertEncoding when the encoding of a string is known and you want to modify its bytes so the same text is represented in a different encoding. DefineEncoding is used to tell the framework what encoding those particular bytes represent. In your case, you might do something like this:

data = newData.DefineEncoding( Encodings.UTF16LE )
data = data.ConvertEncoding( Encodings.UTF8 )
7

Thank you! Good Sir! Perfect