Post a Reply
481 views

BinaryStream Data Corruption

  1. 8 months ago

    Denise A

    is not verified 12 Feb 2018

    Hi. I have a Windows user in Korea that is having trouble opening a binary project file my app creates. I use the code below to create and open and it works fine for everybody else but for some reason the data seems to be corrupted for him.

    I have no trouble opening the data he saves on my Windows machine and have debugged various ways with him testing to see how it is getting corrupted but have no idea.

    Could it be a LittleEndian / BigEndian issue?

    // Save

FileData = ConvertEncoding(FileData, Encodings.UTF8)

    binary = BinaryStream.Create(Document, True)
  
  If binary <> Nil then
    binary.WriteInt32 lenB(FileData)
    binary.Write FileData
    binary.Close
  End If

// Open
binary = BinaryStream.Open(Document, False)

If binary <> Nil then
  
  Do Until binary.EOF
    i = binary.ReadInt32
    FileData = FileData + binary.Read(i)
  Loop
  
  binary.Close

FileData = DefineEncoding(FileData, Encodings.UTF8)
  
End If

  2. Björn E

    12 Feb 2018 Pre-Release Testers, Xojo Pro Iceland

    Are you sure this here was valid?

    FileData = ConvertEncoding(FileData, Encodings.UTF8)

    As in did FileData have encoding at all before converting ? Or was it binary data or text without encoding ? (Either case then that would be the cause right there)

    And no it could not be big vs little endian issue since that happens between Intel and PPC CPU's and PPC CPU's dont exist any more.

  3. Denise A

    is not verified 12 Feb 2018

    Hi Bjorn. The FileData is UTF8 and says so in the Xojo Debugger before the ConvertEncoding call but I just added it before the file save just to be sure. Is it possible that a string could somehow accidentally have mixed encodings, perhaps mostly UTF8 and an unknown nil encoding string part of it but because the majority of it is UTF8 Xojo thinks it's UTF8? If so, then the ConvertEncoding wouldn't correctly work or would force the wrong encoding and cause the data corruption issue.

  4. Jeff T

    12 Feb 2018 Midlands of England, Europe

    Pretty sure Xojo always suggests UTF8 for internal string variables.
    In Korea, if the user created the text, it could easily be in UTF-16

    As far as I can tell, UTF-8 requires 3 bytes per Asian character, and UTF-16 only requires 2 bytes per Asian character.

  5. Derk J

    12 Feb 2018 Pre-Release Testers, Xojo Pro
    Edited 8 months ago

    Did you set the right binarysteam.Littleendian = True/False?

    Try littlendian = True

  6. Greg O

    13 Feb 2018 Xojo Inc

    @Denise A The FileData is UTF8 and says so in the Xojo Debugger before the ConvertEncoding call but I just added it before the file save just to be sure.

    Don’t do that without checking if it’s already UTF-8. You could actually cause issues for yourself. Wrap that code like this:

    if filedata.Encoding <> Encodings.UTF8 then
    FileData = ConvertEncoding(FileData, Encodings.UTF8)
End If

  7. Thomas E

    13 Feb 2018 Pre-Release Testers

    @Greg OLone Don’t do that without checking if it’s already UTF-8. You could actually cause issues for yourself. Wrap that code like this:
    if filedata.Encoding <> Encodings.UTF8 then FileData = ConvertEncoding(FileData, Encodings.UTF8) End If

    huh? I thought it is a NOP when I convert UTF8 Text to UTF8…
    Can you elaborate?

or Sign Up to reply!