Advice on coding etiquette on waiting to do the next step

Hello XOJO community,

I’m looking for some advice. I’m communicating with an Arduino using Serial. I have four stepper motors connected to the Arduino. I send the Serial code to the Arduino formatted X-1-200 (X stepper - 1 Forward - 200 steps), once the Arduino is finished it sends XOJO a serial command “Done Moving X”.

I’d like to run a list of commands (Z-1-3500, Y-0-28500, Y-1-22400) back to back. However, I don’t know the best way to wait for the finished command before executing the next command. Currently, I have the list of commands in the SerialController DataAvalible event handler with a label control that acts as a conditional flag. It works but I think it’s sloppy. Any advice will be appreciated.

[code]//#-------------------- Drop Needle --------------------
if streamIn.left(13)=“Done Moving X” and Window1.Banner.text =“Vial Loaded” then
If SerialController.Open Then
Window1.Banner.text = “Dropping Needle”
SerialController.Write(“Z-1-3500”) // Sample Needle Stepper
Else
MsgBox(“The serial port could not be opened.”)
End If
end

//#-------------------- Draw Sample (28 mL) --------------------
if streamIn.left(13)=“Done Moving Z” and Window1.Banner.text = “Dropping Needle” then
If SerialController.Open Then
Window1.Banner.text = “Draw Sample (28 mL)”
SerialController.Write(“Y-0-28500”) // Sample Syringe Stepper
Else
MsgBox(“The serial port could not be opened.”)
End If

//#-------------------- Rinse Sample to Waste (-3 mL)
if streamIn.left(13)=“Done Moving Y” and Window1.Banner.text = “Draw Sample (28 mL)” then
If SerialController.Open Then
Window1.Banner.text = “Rinse Sample to Waste (-3 mL)”
SerialController.Write(“Y-1-22400”)
Else
MsgBox(“The serial port could not be opened.”)
End If
end
[/code]

Using a state machine like this is the best approach. You could clean it up a little by putting your commands into an array and just execute the “next one” until the array is empty.

if streamIn.left(11)="Done Moving" and ubound(command)>= 0 then
  If SerialController.Open Then
    SerialController.Write(command(0)) 
    command.remove(0) 
  Else
    MsgBox("The serial port could not be opened.")
  End If
end

@Tim Hare 's advice is spot on - in the case of a linear state machine it’s great to use an array. Unfortunately most of the time my state machines are not linear, and I have to fill my dataAvailable events with a bunch of logic to decide what state to transition to based on the prior state.

I also agree it feels sloppy, but I haven’t figured out a better way to deal with it (other than pushing the actual decision-making code out of the dataAvailable event into a state machine class - but it all has to start with DataAvailable).

You can build your commands around processor subclasses that evaluate the feedback, then install the next processor to send the next command, evaluate the next input, etc.

Or some combination.

Hi Kem, could you explain ‘Processor Subclasses’ or link me to an example. Thanks

Yes, but it’s advisable to move the logic out of the dataAvailable event by using a timer. DataAvailable should fire a timer (delay itself is unimportant) and the parsing logic then goes in the timer’s Action event.

I agree, the parser/state machine tends to becomes unwieldy; I eagerly await Kem’s elaboration on “Processor Subclasses” :slight_smile:

I have never heard this. Please elaborate on why it’s advisable, thanks!

It was suggested by Greg O’Lone in another long thread about serial comms; I think if the DataAvailable event gets too busy it can miss incoming bytes and not fire.

Mentioned in this thread: https://forum.xojo.com/43176-serial-communication-in-a-sequential-way

I subclass Serial and give it a MessageQueue() property, a ResponsePending boolean property, and a Dispatch Timer. To send something, you just append (or “addrow”, if you’re an API 2 kind of person) a message to the queue. The dispatch timer is free running, and when it fires it checks ResponsePending; if ResponsePending is false it calls SendNext, which pulls the next message off the queue, deletes it from the queue, and transmits it. When a message is transmitted, ResponsePending is set, and it only gets reset once DataAvailable and the response parser have handled the response, or a timeout occurs. I tried it without the dispatch timer, simply calling SendNext as soon as the state machine was done, but it never worked reliably, maybe due to some race condition (felt like that anyway). Using a dispatch timer settled everything down and now the messages go in and out smoothly in a stately stream.

This is true if that mass of code in the DataAvailable event does any appreciable amount of work. I find that most of that code is branch conditions that determine the appropriate next step and execute fairly quickly. But yes, I do move all the heavy lifting to a timer.

Although I’m doing it, I don’t really understand it. Is the timer’s Action event on a different thread than that of the DataAvailable event?

I think my original problem was due to transmitting new messages while responses to old messages were coming in - incoming bytes were definitely being lost, and I believe this is a low-level bug in the Serial object that has existed since RealBasic days. It’s possible that the solution was my use of a “dispatch timer” scheme to prevent overlap, but by then I’d already moved the parser to a timer event so I don’t know for sure whether that played a role in solving the problem or not.

I don’t have an example, but this is a concept based on design patterns. It’s a way of encapsulating the code into subclasses that can process the input and decide what to do next. If we call that a processor, one of its actions might be to choose which processor it should install to handle the next input.

The alternative is a giant SELECT statement, or similar, that can get unwieldy.

That would be me! :smiley:

Fast handling of the data still requires having as little processing and code being run as part of data available
Basically the data available event should just grab whatever is there and append it to a buffer
It may also start a timer if the timer is not already running
Then the action event of that timer grabs whatever data (how to frame each chunk for processing is up to you or maybe the protocol your implementing) and it then processes one, or more, chunks
The dataavailable event should not do this because if the data processing takes a long time the buffers in use can overflow and you can drop data

And me too @Julia Truchsess . These things usually start out small, so you do the simplest thing. It’s only later, when you see the monster you’ve created, that you consider refactoring.

[quote=479017:@Norman Palardy]Fast handling of the data still requires having as little processing and code being run as part of data available

The dataavailable event should not do this because if the data processing takes a long time the buffers in use can overflow and you can drop data[/quote]
But exactly how does moving the handling code to a timer event solve the buffer overflow issue?

That’s a fair question since Timers run on the Main thread, same as (presumably) DataAvailable. If the thread is tied up, it’s tied up, doesn’t matter where.

The code that connects Xojo to the serial port is interrupt driven - and its running in C++ land
And it may well also be spun off into a background preemptive thread internally - that I am not certain of but do know that this is done frequently for things like database access etc so they do not block on long running queries.
So while YOUR code may not be multithreaded or preemptive thats not to say that Xojo could not be doing that internally.

Moving the data out of the internal buffers to ones you maintain allows the internal buffers to be cleared
There are at least 2 (one in the OS and one in the internal Xojo code for serial port)
The most important to clear is the one at the OS level

[quote=479023:@Norman Palardy]The code that connects Xojo to the serial port is interrupt driven - and its running in C++ land
And it may well also be spun off into a background preemptive thread internally - that I am not certain of but do know that this is done frequently for things like database access etc so they do not block on long running queries.
So while YOUR code may not be multithreaded or preemptive thats not to say that Xojo could not be doing that internally.[/quote]

Yeah, I assumed all that.

[quote=479023:@Norman Palardy]Moving the data out of the internal buffers to ones you maintain allows the internal buffers to be cleared
There are at least 2 (one in the OS and one in the internal Xojo code for serial port)
The most important to clear is the one at the OS level[/quote]

Sorry for being dense, but I still don’t understand how moving the parsing code to a timer event “allows the internal buffers to be cleared”. Yes, I’ve assigned the value returned by ReadAll to a property, but I haven’t explicitly released or cleared anything. If the act of calling ReadAll were sufficient then there’d be no reason to not put code in the event. How does the event know that I’m now running in the timer event?

when you get the data available event you do a readall
This makes sure you grab everything in Xojo internal buffers so you dont miss it
Internally Xojo knows this has occurred, since you called readall, and could then clear its internal buffer that the interrupt code can fill up again
IF you delay handling of the data by putting long running code in DataAvailable then the interrupt code cant put more data into the internal buffer - so you could miss interrupts and therefore drop data

Readall IS almost sufficient in and of itself - but you need to put that data “somewhere” so it can be processed - which is your codes internal buffer. And you dont want to just blindly call readall as Xojo’s internal buffer could be empty

Without exposing internal implementation detail (which I’m not permitted to do) thats about as much as I can tell you