Problem with Serial Read

Good Day,

I’m having a very strange problem with Serial reception. I’m on 2019r1. This code works perfectly 99% of the time, but then after some time running, it will suddenly throw a OutOfBoundsException.

Sub DataAvailable()
  static VRef_Read as New MemoryBlock(100)
  static VRef_Step as integer
  Dim DEBUG1_PreRead, DEBUG1_PostRead as integer
  Dim DEBUG1_DataAvailableTotalBytes as integer
  
  DEBUG1_DataAvailableTotalBytes = Serial_VRef.BytesAvailable
  
  While (Serial_VRef.BytesAvailable > 0)
    DEBUG1_PreRead = Serial_VRef.BytesAvailable
    Dim InByteTemp as MemoryBlock = Serial_VRef.Read(1)
    DEBUG1_PostRead = Serial_VRef.BytesAvailable
    VRef_Read.Byte(VRef_Step) = InByteTemp.Byte(0)
    VRef_Step = VRef_Step + 1
    if VRef_Step > 99 then VRef_Step = 0
  Wend

Essentially, DEBUG1_PreRead gets set to 1, DEBUG_PostRead gets set to 0 - which certainly indicates that I read a byte from the buffer, but InByteTemp is a memory buffer with size of zero. Any ideas here? I’m still ‘fairly’ new to Xojo, but I’ve done a lot of serial comms with it, and this is the first time I’ve seen this problem. One thing to note, this is the first project I’ve done that uses multiple (3) serial ports simultaneously. Essentially capturing a bunch of data from multiple pieces of test equipment.

Any help would be much apprecaited.
Ben

Here are some screen-shots

To add, I know I could simply check the MemoryBlock size before reading it to prevent the error, but then I’ll be missing a byte and throw out a packet for no particularly good reason.

I really need to understand how a Buffer of 1 Byte gets read, leaving the Buffer with zero bytes, but no bytes are actually read.

Also, another interesting point - No matter how many bytes are available when the “DataAvailable” event fires (14 in the case of the screen shot linked), this error ALWAYS occurs on the 1->0 transition.

InByteTemp is a memoryblock that only ever has 1 byte as you read(1) it could even become Nil cause if you don’t have data then the mb could be nil. Nil won’t be given cause the read(1) returns a string so the mb size would be 0 if there are no bytes avail.

The debugger shows your mb size is 0. You didn’t check if the read(1) returned and empty string…

Edit remove incorrect index referencing

How can InByteTemp be a Nil MemoryBlock when

  • There was one byte available (PreRead)
  • I Read(1) the Buffer into InByteTemp
  • There is then zero bytes available (PostRead)

That’s the problem. I obviously read a byte, but InByteTemp is Nil. That shouldn’t be.

The Index is 31 for a different MemoryBlock having nothing to do with InByteTemp.

Nil won’t be given by read(1) as it returns a string, strings can’t be nil.

This is probably a bug as there was 1 byte avail and you have read it (the count went down).

It seems like a race-condition where read couldn’t return cause the value from the buffer wasn’t set yet. Probably cause your loop is tight (single byte repeating)

Make a bugreport and add link to this post.

Will do.

A quick fix could be to check for packet size then read the entire packet.

Or read ALL data then buffer it in a string then parse from the bufferString.Bytes (count).

You’ll be sure you can loop then.

Variable Length Packet, so the first solution is difficult.

Reading to a String just seems wasteful since I’m working with Bytes. There is literally nothing human readable about these transmissions.

I’d rather get to the bottom of this and fix it.

Well a string and memoryblock are interchangeable so don’t worry. If you read the entire buffer in a string and do

AString = AMEmoryblock
AmemoryBlock = AString

It’s almost the same

Mystring is a property of the class
Here is something out of my head:

Mystring = mystring + ser.ReadAll(Nil)
While Mystring.bytes > 0
// do your stuff...
Var tmpbm As memoryblock = mystring.LeftBytes((1)
// do something with your mb
Wend
1 Like

But sice you can get a DataAvailable event per say 100 bytes (so 1 event has maybe alot of bytes in the read buffer) for example your loop will choke xojo’s event loop for too long causing your issue.

Interesting Theory. I’ll try doing a ReadAll to a memory block and then quickly parsing that based on variable length.

1 Like

Try not to use the serial read calls in the loop. Only amb.size or astring.bytes for the lengths for example. don’t use bytesavailable etc…

Speed will also improve

1 Like

I removed the loop entirely and changed to ReadAll. Still seeing the problem, though less often (This time it took a few million bytes before the problem showed up, whereas it could be readily duplicated with only a few tens of thousand before)

Sub DataAvailable()
  static VRef_Read as New MemoryBlock(100)
  static VRef_Step as integer
  Dim DEBUG2_PreRead, DEBUG3_PostRead as integer
  Dim DEBUG1_DataAvailableTotalBytes as integer
  Dim DEBUG4_ErrorDifference as integer
  
  DEBUG1_DataAvailableTotalBytes = Serial_VRef.BytesAvailable
  
  If Serial_VRef.BytesAvailable > 0 Then
    DEBUG2_PreRead = Serial_VRef.BytesAvailable
    Dim InByteTemp as MemoryBlock = Serial_VRef.ReadAll
    DEBUG4_ErrorDifference = DEBUG2_PreRead - InByteTemp.Size
    DEBUG3_PostRead = Serial_VRef.BytesAvailable
    
    if DEBUG4_ErrorDifference <> 0 Then
      DEBUG4_ErrorDifference = DEBUG4_ErrorDifference 'Junk just to give me something to break on
    end if
    
  End if

I set a break point Inside of the “if DEBUG4_ErrorDifference <> 0 Then” condition.
When it broke, the DEBUG4_ErrorDifference was 16 and PreRead was 16. This indicates that there were 16 bytes in the buffer, but I ended up with a Nil MemoryBlock.

More weirdness. LookAhead works correctly, so there really are bytes there, and they really do get dropped. This seems like a pretty major bug.

Sub DataAvailable()
  static VRef_Read as New MemoryBlock(100)
  static VRef_Step as integer
  Dim DEBUG2_PreRead, DEBUG3_PostRead as integer
  Dim DEBUG1_DataAvailableTotalBytes as integer
  Dim DEBUG4_ErrorDifference as integer
  Dim DEBUG5_PreReadAgain as integer
  DEBUG1_DataAvailableTotalBytes = Serial_VRef.BytesAvailable
  
  If Serial_VRef.BytesAvailable > 0 Then
    DEBUG2_PreRead = Serial_VRef.BytesAvailable
    Dim Junk as MemoryBlock = Serial_VRef.LookAhead
    DEBUG5_PreReadAgain = Serial_VRef.BytesAvailable
    Dim InByteTemp as MemoryBlock = Serial_VRef.ReadAll
    DEBUG4_ErrorDifference = DEBUG2_PreRead - InByteTemp.Size
    DEBUG3_PostRead = Serial_VRef.BytesAvailable
    
    if DEBUG4_ErrorDifference <> 0 Then
      DEBUG4_ErrorDifference = DEBUG4_ErrorDifference 'Junk just to give me something to break on
    end if
    
  End if

Sorry for the multiple posts for the images - New users can’t attach more than one link to each post.

So basically

  • BytesAvailable = 20
  • LookAhead returns 20 Byte MemoryBlock
  • BytesAvailable still 20
  • ReadAll Returns Nil MemoryBlock
  • Bytes Available = 0

Not pretty.

Also - Can someone from the Xojo forum here allow me to post more replies as a new user? Apparently this is the last response I can make today.

You dont get a NIl memoryblock but a memoryblock with 0-size.

You should check the size, that’s one Thing

Is this the “Serial” class or “SerialConnection” class?

Maybe something else is going on:
https://documentation.xojo.com/api/language/readable.html#readable-read

“if not enough memory is available you may get a empty string”
It “could” be ?

1 Like

Sorry for the delay. Had to wait 24 hours to post any more responses.

Serial Class (2019r1)

Well ok, MemoryBlock with zero size. Either way, essentially Nil when it should be filled with data.

What good does checking the size do, or more particularly, why should I need to check it? The size should be equal to BytesAvailable. If BytesAvailable > 0, then I should NEVER get a size zero MemoryBlock as a return. I’m losing data. .LookAhead proves that there was data in the buffer that I do not get with .ReadAll, but that .ReadAll clears that data from the buffer. Checking the size will prevent a crash, but it doesn’t change the fact that those bytes are gone forever. That’s the ultimate problem.

Memory being available is not a problem. I have 12 Gigabytes (physical) free and I’m trying to read 20 bytes. This is a small Xojo app - it’s not like I’m pushing the limits of anything here.