Problems with serialConnection.ReadAll

Hi,
I am working on moving a Xojo project for controlling a lab instrument via serial commands via USB and am having lots of problems. This is a project that I have developed over many years and have successfully adapted to several different instruments of the same general class, including the one I am working with now. But, now I am trying to get the project to work on Windows and am encountering problems with the serial communication.

The general scheme is that the computer sends a command to the instrument, and a reply is sent back. Here is an example, as seen from a terminal running on Windows:

Sent command:
GOTO
Received string:
GOTO
GOTO=405.0

The command is executed with a carriage return character, but the lines returned are separated by line-feed characters sent by the instrument. The responses from all of the commands end with the “#”.

Here is the code that I have written to send the command and read the reply:

mySerial.Write(‘GOTO’+CR)
mySerial.XmitWait

time = ticks
while not (Right(mySerial.LookAhead,1) = “#”) and ticks < time + 120
mySerial.Poll
wend
retStr= mySerial.ReadAll
messageBox(retStr)
retStr = retStr.Right(8)
retStr = retStr.Left(3)

The basic scheme is to monitor the serial buffer (for upt to 20 seconds) until the “#” character appears, and then read the buffer. The problem is that the result from the previous execution of a command seems to still be in the buffer, even though it was read with a readall command. My understanding is that readall is supposed to clear out the buffer.

I have tried things like adding delays and extra readall commands to make sure that the buffer is flushed, but nothing seems to help (or at least not in a consistent way).

Any suggestions would be greatly appreciated!
David

I always use the DataReceived event as opposed to polling. In DataReceived I use ReadAll to append the port’s buffer contents to a buffer string property of my SerialConnection subclass and Timer.CallLater or a one-shot timer with a short period to call my parsing method, to ensure data parsing and handling is on the main thread and doesn’t bog down the port’s low-level code.

In the parsing method I use regex to look for a valid message. If found, I set the buffer string property to null (“”) and raise an event to handle the message, else I just return.

There have been some forum topics lately about the latest Xojo firing DataReceived when there’s not actually any received data, but I don’t experience that with 2021r2.1.

MBS’s new SerialPortMBS classes are good and slightly better organized than Xojo’s imo, FWIW.

Julia,
Thanks very much for your quick reply. I’m afraid that much of what you have suggested goes over my head! Could you point me to a code example?
Thanks again,
David

So Julia being kinda Green at this - When do you decide to upgrade to different versions. I seem to get bit on having stuff (Serial - DataReceived - Firing when nothing is available) not working the same.

I first subclass SerialConnection and add a string Property RXBuffer, a Timer property PollTimer (not a great name), and an event “SensorDataAvailable” to the subclass. Here’s the Constructor

Public Sub Constructor()
  // Polltimer is a one-shot, fired by DataAvailable, and serves simply to get the buffer parsing code out of the
  // DataAvailable event.
  
  PollTimer = new Timer
  PollTimer.Period = kPollTime
  PollTimer.Mode = timer.ModeSingle
  PollTimer.Enabled = True
  AddHandler PollTimer.Action, WeakAddressOf ParseRX
  
End Sub

Here’s the code in the DataReceived event:

Sub DataReceived() Handles DataReceived
  // The point of the poll timer is not to delay, so much as it is to get the parsing code out of the DataAvailable event. (could probably use Timer.CallLater instead of a dedicated timer?)
  
  RXBuffer = RXBuffer + Me.ReadAll
  PollTimer.Reset // PollTimer.Action is handled by ParseRX
End Sub

And here’s the code in ParseRX:

Private Sub ParseRX(Sender As Timer)
  // This is the handler for PollTimer, which is a one-shot fired by our DataReceived event.
 
  Dim rgx As New RegEx, rgm As RegExMatch
  
  rgx.SearchPattern = myRegexSearchPattern
  rgm=rgx.Search(RXBuffer)
  
  If rgm <> nil then
    
    RXBuffer = ""
   RaiseEvent SensorDataAvailable(Asc(rgm.SubExpressionString(1))) // Whatever payload you extract with the regex

    End
  
End Sub

I’m stuck at 2021r2.1 at present because I have way too much API 1 legacy code to bother updating it to API 2 until and unless I absolutely have to. Not a big fan of all the DesktopXXX stuff :frowning:

1 Like

Hi David,

I can’t help you as I have never tried to control an instrument directly with Xojo (I have interfaced an RB app with a LabView app which was controlling an instrument many yeas ago), but I am curious about what instrument you are trying to interface with, if you don’t mind sharing.

Thanks,
Karen (who spent a lot of years as an Analytical Chemist)

Karen,
The instrument is a simple UV-Vis spectrophotometer, and the primary function of the program is to record absorbance values at intervals, and control the instrument. The particular instrument I am working with now is a ThermoScientific Genesys 10s. This was really straightforward when there were serial ports on computers and instruments, at least on the Mac. But, with USB things has become almost impossible on the Mac, which is why I am trying to make it work on Windows. I’m not sure why, but timing issues seem to be much trickier on Windows.

David

Jill,
Thanks very much for the sample code!
David

Needless to say be careful with the spec. Send a few wrong commands and something really bad and expensive could happen.

Is it SCPI, or something else? I have some code for controlling instruments via SCPI.

No, it is something different and proprietary, and much simpler, I think.

Thanks for the warning, and I’ve never given that any thought. This is a very simple instrument with a limited command set that just gives access to standard user functions. It could be that there are additional commands that could affect the instrument at a deeper level, but those kept secret, I think.
David

1 Like

Julia.
Thanks for this. I’m afraid, though, that I’m pretty ignorant of how to implement all of it. Could you give me a bit more guidance? Here is what I have done so far:

  1. Created a subclass of SerialConnection, called SerCon1, which is located outside of any other objects.
  2. Added a Constructor method to SerCon1, with the code you provided.
  3. Added a string property, RXBuffer to SerCon1.
  4. I’ve added the code you provided to the DataReceived event handler for SerCon1.
    Am I OK so far?
    Now, for some questions:
  5. I’ve not added PollTimer directly to SerCon1, since I presume that the Constructor does that. Is that correct?
  6. How do I add the SensorDataAvailable event? I see that I can add an event definition to SerCon1, but how is the event defined, and what goes in its handler?

At an even more basic level, once I have everything set up with SerCon1, how do I interface this with the rest of the project?

I apologize for these very basic questions! Your help is greatly apprecited!
David

Your subclass needs a property of type Timer, named PollTimer. The Constructor instantiates the timer, but it needs a property to assign the instance to.

Yes, add an event definition, with parameter(s) to pass the value(s) received. Also visible in the screenshot above.

Here’s where the magic happens :slight_smile: Assuming your instrument is associated with a window, drag a Generic Object from the LIbrary onto the window, set its super to whatever you named your subclass, and give it a name by which it will be known and seen in the Controls section of your window’s Navigator:

Screenshot 2024-01-04 at 12.51.32 PM

Now in your window’s Navigator, select Add Event to the control and add the SensorDataAvailable (or whatever name you gave in the event definition):

and that’s where your window code goes. In my case, I prefer to put code in window methods rather than in event handlers, so my event handler just calls a window method HandleFixtureData.

If your instrument is not associated with a window, you’ll want to do something like add a property of your subclass’s type to a module, and when your app opens instantiate it with New and add a handler in code to its SensorDataAvailable event using AddHandler.

I very much expect this test is your problem

while  not (Right(mySerial.LookAhead,1) =  "#")

You are checking the last byte in the input buffer, which will only work as intended when bytes arrive in the buffer one at a time. Between the serial port and your Xojo app is a kernal driver and a somewhat clunky Windows thread scheduler. The outcome is the input stream may arrive at your Xojo app in arbitrary slices.

For instance. Let’s say your instruments sends the string “RESPONSE1#[LF]RESPONSE2#[LF]” and it takes four iterations of your loop to arrive. On each iteration the input buffer might look something like this.

1. RESP
2. RESPONSE1#[LF]RES
3. RESPONSE1#[LF]RESPON
4. RESPONSE1#[LF]RESPONSE2#[LF]

The following code parses a serial stream delimited by line feeds into an array.

Const TIME_OUT As Integer = 20000
Const REQUEST As String = "GOTO"

Var pos, start, now As Integer
Var response() As String
Var done as boolean

Call mySerial.ReadAll()       //clear detritus from the input buffer
mySerial.Write(REQUEST + EndOfLine.CR)  //send the command

start = System.Microseconds / 1000
Do
 mySerial.poll
 now = System.Micrososeconds / 1000
 pos = mySerial.LookAhead.IndexOf(EndOfLine.LF)
 If pos > -1 Then
   response.Add mySerial.Read(pos +1)
   done = (mySerial.LookAhead.IndexOf(EndOfLine.LF) = -1)
 End If
Loop Until done or (now - start > TIME_OUT) 

For Each s As String In response
  TextArea1.AddText s + EndOfLine
Next
1 Like

Excellent point, but the OP states that his issue is that the buffer isn’t being cleared by ReadAll after he gets the response, not that he’s not getting the response.

1 Like

Matthew,
Thanks very much for your suggestions. I’ve tried what you suggest, and it hasn’t helped. The # character should be the very last thing sent from the device, and messages are only sent after a prompt from the program, so looking for the # should work, even if the message comes in blocks.
There is something very odd going on, with results from a previous prompt showing up in response to the current one. I’m starting to wonder if the problem is with the signal being sent to the device more than once, rather than in the handling of the reply.
In any case, I don’t think that I am going to be pursuing this much further, at least not for a while.
Thanks again,
David

Dear David, the main problem on Windows is that commands to do anything with the serial bus do not execute until the entire method has completed. In other words, at the point in your code where you wrote that the serial port should write/send something, it isn’t going to do that exactly at that moment. So your next line of code doesn’t do anything, and the following code can’t be considered to be executing after your data has been sent, because it isn’t. Why Xojo works this way I have no idea, it’s maddening. What you have to do is encapsulate any “write” commands and use events to control what needs to happen. In other words, on windows you simply can’t work logically in terms of code executing from top to bottom, it just doesn’t work that way. Totally counter-intuitive.

All of the code after the write command is useless because the write isn’t going to happen until the method is over. On Mac this works as expected. On Windows, nope.