New Framework - TextEncoding from Byte Order Mark

General
1

Hi everyone,

i try to use the new framework all the way. I don’t have experiences using Xojo.IO.BinaryStream. What do i want? The User can select via Dialog any Textfile and load it into a TextArea. I don’t know the Xojo.Core.TextEncoding of the file. So using this:

[code]Using Xojo.Core
Using Xojo.IO

Dim f As FolderItem
f = SpecialFolder.Documents.Child(“SaveData.txt”)

If Not f.Exists Then
’ Cannot read the file if it does not exist
Return
End If

Dim errorText As Text
Try
Dim input As TextInputStream
input = TextInputStream.Open(f, TextEncoding.UTF8) // don’t know the Encoding
TextFileArea.Text = input.ReadAll
input.Close
Catch e As IOException
errorText = "File IO Error: " + e.Reason
End Try[/code]
will not be an option. First i need to check the BOM (Byte Order Mark). I thought writing a Function returning Xojo.Core.TextEncoding would be the best. Can someone please help me to write such a Function?

[code]// Pseudo
Using Xojo.Core
Using Xojo.IO

Dim f As FolderItem = SpecialFolder.GetResource(“SaveData.txt”)

Dim b As BinaryStream
b = BinaryStream.Open(f, BinaryStream.LockModes.Read)

// check Byte Order Mark
// ASCII BOM ???
// UTF-8: EF BB BF
// UTF-16 (BE): FE FF
// UTF-16 (LE): FF FE
// UTF-32 (BE): 00 00 FE FF
// UTF-32 (LE): FF FE 00 00
Dim enc TextEncoding

// ???

b.Close

return enc[/code]
Thanks for your input using BinaryStream etc. and time.

2

After few hours, i got a working solution. :slight_smile: It uses the new Xojo-Framework.

[code]Dim f As FolderItem = GetOpenFolderItem("")

If f <> Nil Then

Try

Dim openFile As New Xojo.IO.FolderItem(f.NativePath.ToText)
Dim b As Xojo.IO.BinaryStream

b = Xojo.IO.BinaryStream.Open(openFile, Xojo.IO.BinaryStream.LockModes.Read)

// first, check for a BOM

Dim startPosition As UInt64 = b.Position
Dim b1 As UInt8 = b.ReadUInt8
Dim b2 As UInt8 = b.ReadUInt8
Dim b3 As UInt8 = b.ReadUInt8
Dim b4 As UInt8 = b.ReadUInt8
Dim enc As Xojo.Core.TextEncoding

If b1 = &hEF And b2 = &hBB And b3 = &hBF Then
  
  // UTF-8
  enc = Xojo.Core.TextEncoding.UTF8
  
ElseIf b1 = &hFF And b2 = &hFE And b3 = &h00 And b4 = &h00 Then
  
  // UTF-32 (LE)
  enc = Xojo.Core.TextEncoding.UTF32LittleEndian
  
ElseIf b1 = &h00 And b2 = &h00 And b3 = &hFE And b4 = &hFF Then
  
  // UTF-32 (BE)
  enc = Xojo.Core.TextEncoding.UTF32BigEndian
  
ElseIf b1 = &hFF And b2 = &hFE And b3 <> &hFE And b4 <> &hFF Then
  
  // UTF-16 (LE)
  enc = Xojo.Core.TextEncoding.UTF16LittleEndian
  
ElseIf b1 = &hFE And b2 = &hFF And b3 <> &hFE And b4 <> &hFF Then
  
  // UTF-16 (BE)
  enc = Xojo.Core.TextEncoding.UTF16BigEndian
  
Else
  
  // reset BinaryStream
  b.Position = startPosition
  
  // no BOM; see if it's entirely ASCII.
  Dim mb As New Xojo.Core.MemoryBlock(b.Read(b.Length))
  Dim i As Integer
  Dim maxi As Integer = b.Length-1
  
  For i = 0 To maxi
    
    If mb.UInt8Value(i) > 127 Then Exit
   
  Next
  
  If i > maxi Then
    
    // ASCII
    enc = Xojo.Core.TextEncoding.ASCII
    
  Else
    
    // Default: UTF-8
    enc = Xojo.Core.TextEncoding.UTF8
    
  End If
    
End If

b.Close
  
// open Textfile
Dim input As Xojo.IO.TextInputStream

input = Xojo.IO.TextInputStream.Open(openFile, enc)
TextArea1.Text = input.ReadAll
input.Close

Catch e As IOException
MsgBox "File IO Error: " + e.Reason
End Try

End If[/code]