Why No BinaryStream.WriteCstring or ReadCString?

  1. 9 weeks ago

    Karen A

    Jun 17 Pre-Release Testers
    Edited 9 weeks ago

    There are MemoryBlock.CString(offset) setter and getter methods, but why are there no BinaryStream.ReadCstring and WriteCString methods? Was it an oversight or is there a reason for not having them? If there is no reason not to have them I'll put in a feature request

    Writing an Extends method for BinaryStream.WriteCstring method is simple and would be consistent with the rest of the API (Just write the string and then a Null byte)... But doing ReadCString as an extends would mean (as far as I can see) doing something like read the stream byte by byte until i find a 0 (or reach the end of the MB) - which could be slow...

    The better way would likely be to work with the memoryblock and do something like:

    Dim S as string = MB.CString(Stream.Position)
    Stream.Postion = Stream.Postion + S.LenB + 1

    So I can't extend BinaryStream unless I also pass in the memoryblock... doable but not consistent with the API...

    It really feels like this capability should be built in!

    - Karen

    Cstrings were never meant for general use. They’re really only there for use with declares.

  2. Karen A

    Jun 17 Pre-Release Testers

    Submitted a feature request:

    Feedback Case #56095

  3. Greg O

    Jun 17 Xojo Inc Answer

    Cstrings were never meant for general use. They’re really only there for use with declares.

  4. Karen A

    Jun 17 Pre-Release Testers
    Edited 9 weeks ago

    @Greg OLone Cstrings were never meant for general use. They’re really only there for use with declares.

    Is not the same true for PStrings for which there are BinaryStream methods?

    CStrings are useful for compact binary data formats... else one needs either to use fixed string sizes (padded with nulls when needed) or write longer strings analogously to PStrings where one uses more than 1 byte for string length.

    Initially I was writing code to do the latter. but as i'm trying to be as compact as possible, I realized Cstrings only have 1byte overhead regardless of size- though I suspect with the penalty of more CPU usage for reading. I was surprised to find those methods were missing!

    Anyway in the spirit consistency and completeness please consider adding those methods as they really are useful for things beyond declares.

    - Karen

  5. Jean-Yves P

    Jun 17 Pre-Release Testers, Xojo Pro Europe (France, Besançon)

    to write long PStrings, I ended up with this :

    Public Sub WriteString(extends bs As BinaryStream, s As String)
      If s.LenB<256 Then
        bs.WriteUInt8(1)
        bs.WriteUInt8(1)
        bs.WritePString(s)
      Elseif s.LenB<65536 Then
        bs.WriteUInt8(1)
        bs.WriteUInt8(2)
        bs.WriteUInt16(s.LenB)
        bs.Write(s)
      Else
        bs.WriteUInt8(1)
        bs.WriteUInt8(4)
        bs.WriteUInt32(s.LenB)
        bs.Write(s)
      End If
      
    End Sub

    and this

    Public Function ReadString(extends bs as BinaryStream, Enc as TextEncoding = nil) as String
      Dim detector As UInt8 = bs.ReadUInt8
      
      If detector=1 Then
        Dim psize As UInt8 = bs.ReadUInt8
        
        If psize>0 Then
          Select Case psize
          Case 1
            ' standard 8 bits pstring
            Return bs.ReadPString( Enc)
          Case 2
            Dim size As UInt16 = bs.ReadUInt16
            Return bs.Read(size, Enc).DefineEncoding(Encodings.UTF8)
          Case 4
            Dim size As UInt32 = bs.ReadUInt32
            Return bs.Read(size, Enc).DefineEncoding(Encodings.UTF8)
          Else
            ' still original 8bits pstring
            bs.Position = bs.Position-2
            Return bs.ReadPString( Enc)
          End Select
        Else
          Return ""
        End If
      Else
        ' standard 8 bits pstring
        bs.Position = bs.Position-1
        Return bs.ReadPString( Enc)
      End If
      
    End Function
  6. Norman P

    Jun 18 Pre-Release Testers, Xojo Pro great-white-software.com/blog

    @Karen A Is not the same true for PStrings for which there are BinaryStream methods?

    CString, PString, WString and some others leaked out from being only available for declares years ago (pre-2008 ish)

    As for writing compact binary formats the IDE does that with binary projects and doesnt need or use CStrings
    It basically uses a lengtn/data format like Jean-Yves describes

    For a CString something, whether its your code or framework code, has to read byte by byte until it hits the NUL byte

  7. Karen A

    Jun 18 Pre-Release Testers
    Edited 9 weeks ago

    @Norman P As for writing compact binary formats the IDE does that with binary projects and doesnt need or use CStrings
    It basically uses a lengtn/data format like Jean-Yves describes

    For a CString something, whether its your code or framework code, has to read byte by byte until it hits the NUL byte

    As I said, the approach I originally took was including the length information... The tradeoff between the 2 approaches is processing speed vs compactness... which is more important (and if the difference is significant) depends on specific usage.

    BTW I would think the framework code would be faster at finding the Nul byte than my code could be.

    - karen

  8. Karen A

    Jun 18 Pre-Release Testers
    Edited 9 weeks ago

    @Jean-YvesPochez to write long PStrings, I ended up with this :

    That got me thinking... This code is meant to serialize generic recordsSets which could have a lot records and a lot of fields. Most text strings used as IDs or for descriptions in table fields tend to be shorter than 255 characters, so maybe this would be a little more efficient if a bit less elegant:

    Public Sub WriteString(extends bs As BinaryStream, s As String)
    Const Bit8 = 253
    Const Bit16 = 254
    Const Bit32 = 255
    
    Select Case S.LenB
    Case Is < Bit8
       bs.WritePString S
    
    Case Is < 256
       bs.WriteUInt8 Bit8
       bs.Write S  
    
     Case Is <65536 Then
       bs.WriteUInt8 Bit16
       bs.Write S  
    
      Else
        bs.WriteUInt8 Bit32
        bs.Write(s)
      End Select
      
    End Sub

    and this

    Public Function ReadString(extends bs as BinaryStream, Enc as TextEncoding = nil) as String
    
    Const Bit8 = 253
    Const Bit16 = 254
    Const Bit32 = 255
    
     Select case  bs.ReadUInt8
     Case Is < Bit8
         Return bs.ReadPString(Enc)
      Case Bit8
         Return  bs.Read(bs.readUint8, Enc)
      Case Bit16
         Return  bs.Read(bs.readUint16, Enc)
      Else
         Return  bs.Read(bs.readUint32, Enc)
      End If
    End Function
  9. Norman P

    Jun 18 Pre-Release Testers, Xojo Pro great-white-software.com/blog

    its still much the same loop in the framework or your code

    dim buffer as string
    while binarystream.eof = false
        dim c as Uint8 = binarystream.readUint8
        if c <> 0 then
            buffer = buffer + chrb(c)
       end if
    wend
    
    return buffer
  10. Karen A

    Jun 18 Pre-Release Testers
    Edited 9 weeks ago

    @Karen A That got me thinking... This code is meant to serialize generic recordsSets which could have a lot records and a lot of fields. Most text strings used as IDs or for descriptions in table fields tend to be shorter than 255 characters, so maybe this would be a little more efficient if a bit less elegant:

    Oops that should have been:

    Public Sub WriteString(extends bs As BinaryStream, s As String)
    Const Bit8 = 253
    Const Bit16 = 254
    Const Bit32 = 255
    
    Select Case S.LenB
    Case Is < Bit8
       bs.WritePString S
    
    Case Is < 256
       bs.WriteUInt8 Bit8
       bs.WritePString S  
    
     Case Is <65536 Then
       bs.WriteUInt8 Bit16
       bs.WriteUInt16 S.LenB
       bs.Write S  
    
      Else
        bs.WriteUInt8 Bit32
        bs.WriteUInt32 S.LenB
        bs.Write(s)
      End Select
      
    End Sub

    and this

    Public Function ReadString(extends bs as BinaryStream, Enc as TextEncoding = nil) as String
    
    Const Bit8 = 253
    Const Bit16 = 254
    Const Bit32 = 255
    
     Select case  bs.ReadUInt8
     Case Is < Bit8
         bs.Position = bs.position -1
         Return bs.ReadPString(Enc)
      Case Bit8
         Return  bs.Read(bs.readUint8, Enc)
      Case Bit16
         Return  bs.Read(bs.readUint16, Enc)
      Else
         Return  bs.Read(bs.readUint32, Enc)
      End If
    End Function

    This implementation saves one byte per string for strings > 255 characters

or Sign Up to reply!