Separating commands sent via TCPSocket

General
1

I have a function that fires off 5 commands over a TCPSocket to a motion control unit, to gather some data from its EEPROM:

SendCommand( "[GetDesktopServerIP]" )
SendCommand( "[GetDesktopServerPort]" )
SendCommand( "[GetClearCoreIP]" )
SendCommand( "[GetClearCorePort]" )
SendCommand( "[GetHearbeatInterval]" )

However, this is happening so fast that it appears to be sent in a single packet. This is what the MCU is echoing back as the command it received:

>[GetDesktopServerIP][GetDesktopServerPort][GetClearCoreIP][GetClearCorePort][GetHearbeatInterval]

The problem is that at this time, the MCU can’t handle more than one command at the same time, it only pays attention to the first one, so I need to fire these off quickly, but separately.

I could pause between each one, but I don’t really like doing that if I don’t have to. I have tried calling TCPSocket.flush after each of the above commands but the result is that it works for the first one, but not subsequent responses. That is:

>[GetDesktopServerIP]
[STATUS|Command Successful, Server IP address = 192.168.1.221]
>[GetDesktopServerPort][GetClearCoreIP][GetClearCorePort][GetHearbeatInterval]
[STATUS|Command Successful|Port ID=8889]

Even though the code was modified to flush after each SendCommand call, it looks like it only worked for the first command [GetDesktopServerIP], and the next three got bunched together.

What’s a surefire way to make sure these commands all go out separately? is my only option to build in a pause between each one?

2

None in xojo. This is the way TCP is designed and Xojo uses the OS implementation without offering a way to tune how packets are treated, so unles your pause is in the magnitud of seconds to be sure, the commands will be cached and sent in the same packet. Yo will have to implement a parser in the other end, add all the bytes to a queue, then process each command in FIFO. For thos use cases I use an MQTT implementation

Or try UDP

3

Are you sure there’s not supposed to be a delimiter (like an End-Of-Line) separating the commands?

2 Likes
4

**This is correct. In the DataAvailable event, you should be breaking the string apart into an array by the delimiters, and looping thru the array to handle each command. This will keep your commands in sequential order and ensure none get dropped…

5

We are using [ and ] to indicate the start and end of discrete commands. Right now, the firmware is not set up to accept multiple commands per incoming message. That is, it’s expecting [GetDesktopServerIP] and [GetClearCorePort] to be in independent messages. It’s not expecting [GetDesktopServerIP][GetClearCorePort]. In this second case, it’s throwing out everything after the first “]” – it is not ideal but what we have at the moment.

I’m talking to our firmware developer about parsing the incoming message to handle multiple commands at once. In our testing of individual commands it’s been fine, but now that we have multiple things going at the same time (such as automated pings to check that the MCU is still there), we’re finding that on the MCU side, some instructions are getting lost, because they’re getting piggybacked into the same message as another command.

6

You will need to parse the incoming data, for example

Var commands() as String = me.ReadAll.Split("]")

For i as integer = 0 to commands.lastrowindex
Var currentCommand as String = commands(i).nthfield("[",2).trim

//Now use “currentCommand” to process the command. The for/next loop will automatically move-on to the next command and process it.

next

**TCPSocket does not break up commands, it’s a perpetually connected datastream. It’s the equivalent of pouring marbles into a funnel…you’ll need to catch each “marble” that falls thru the funnel and do something with it. Each marble that falls into the funnel though, will remain in the order it’s sent… the process should be really easy to implement.

7

thanks - but I’m not talking about receiving commands in Xojo - we are sending from Xojo to an external device. The issue is on the device and how it is processing the messages it’s receiving from our Xojo application.

Our firmware developer is about to leave on a month-long trip next week so we’re looking for a solution on the Xojo side but it doesn’t sound like that’s an option. We can’t wait seconds between commands.

8

Should you be getting response data from the motion control unit? Then all you have to do is wait for the response before sending new data.

2 Likes
9

In some cases, commands get responses, but not all. Or at least, not all immediately. For example, I can send commands to move two linear stages into certain positions as separate commands, and the MCU will handle them simultaneously. It may take 20 seconds for each one to get where it’s going. But there’s other stuff that needs to be done in the mean time, so we can’t make everything 100% sequential or we’ll be waiting all day for things to finish before moving on to the next thing.

The bigger issue though, is that there are some commands being sent automatically. For example, there is a periodic “ping” we send to the MCU to make sure it’s still there. If my application happens to send a message at the same time as one of these pings, it goes out as something like:

[ClearCoreServerCheck|81862691][SetDesktopServerPort:8889]`

instead of

[ClearCoreServerCheck|81862691]
[SetDesktopServerPort:8889]

The result in the first case that the command to set the server port is piggybacked on the automated message, and is subsequently lost when it’s received on the MCU. The second case works as expected.

So what I need to do is force the outgoing message to be split on a per-command basis in the short term, and in the long term, the MCU needs to handle the situation in which multiple commands come in at once.

10

Can you open two connections, one for each purpose?

11

I’m not sure I see how that would help. I mean, maybe for the heartbeat checks, but there are other things that can happen concurrently that would be out of my control so while it might fix that specific issue, I can see other cases where multiple commands get sent to the MCU at once. There are quite literally, a lot of moving parts on this machine!

the Firmware developer says this is a major “enhancement”, which is disappointing, because he’s about to leave the country for a month, and we’re already months behind schedule with this project (not because of him).

I can build brief delays into the code if that helps, but not seconds. Are the Xojo docs incorrect in stating that (regarding Flush):

This function can be useful in point-to-point communication over sockets and similar connections: To optimize for transmission performance, some types of output streams try to collect small pieces of written data into one larger piece for sending instead of sending each piece out individually. By calling Flush, the data collection is stopped and the data is sent without further delay, reducing latency.

It doesn’t really seem to work as described. or when it says “the data is sent…” does that mean Xojo is sending the data along to the OS TCP stack, and we have to wait there for it to send?

12

In xojo TCP, the flush is more like a sugestion for the underlying OS

This is the nature of TCP, that parsing should be there from planning. check if you have the right person for the firmware.

Your device could simply answer with a CommandReceived. A responce dont imply the work is done.

Make your firmware developer to add a CommandReceived featur before its vacations.

Implement the queue in the Xojo side. Send all commands to that queue.

When you receive the “CommandReceived” in Xojo, send next command in queue

Add some logic, If queue is empty and if you are not waiting for a CommandReceived send command ASAP if not, add to queue.

13

CommandReceived or an echo back of the command sent is already implemented. So I’ll look into doing a queue on my end.

14

Have you tried to send the message , then flush, then wait the send complete event to send the next one?

15

I haven’t but I’ll give that a try after lunch. sounds similar in principal to the outgoing command buffer that Ivan is suggesting so I’ll have to see which is faster.

As of right now, I’ve determined that adding in delays requires a delay of 400ms after each command, to reliably separate them. That’s unacceptable as there are times when we will need to send 15-20 commands per second.

I don’t see any way of hitting those speeds without the firmware being modified.

16

Adding arbitrary delays is definitively not the way to go.
Ideally, your firmware should handle each command separately (loop thru all commands surrounded by using NthField) so even with commands coming at the same time, it can handle all of them.

1 Like
17

If this is for starting up or so… Why not ask the dev to push something like “[GetCompleteConfig]” ?
Normally in streams or command based packaging you’d have something like is being suggested already, a packet has a strart/end when it’s detected at the device level it’s queued (or buffered) and then processed asap the responses are basicly packets too with start/end.

For now it seems there is no issue? From your quote:

You get the correct sequence, and you have all of them back. The issue is at the device level e.g. it’s sending the echo back and then it’s processing it which is too late. Your device may be behaving better if:

  1. It consumes the data as a buffer
  2. It reads the first from the buffer
  3. write back an echo for the command
  4. process the command
  5. wite back the output of them command
  6. Repeast from step 2 (until buffer is empty, then go to 1).

Now you should get the correct sequence in xojo.
Getting an echo directly with everything you have send is probably not expected on a high end machine (pc).

If you need a way to fix this from the xojo side i’d be sending a command “[GetDesktopServerIP]”, set a string LastCommand = “[GetDesktopServerIP]” in dataavailable check for the echo. if the echo is there, then your dataavailable is waiting for it’s status command when that’s received set LastCommand = “” now your queue sends out the next one. There is not really a timer required but on disconnect and connect/reset etc. i’d be setting lastCommand = “” and clear the buffer e.g.
Call Sock.ReadAll
In the case a response is not expected just set lastCommand = “”

18

Sounds like you need to send one command and then wait for an ACK of that command (i.e., confirmation of its receipt) before proceeding to send the next one. You could include a number string (incremented for each new command sent) as part of the string and the ACK.

1 Like
19

You don’t want to use flush in xojo it could hang (deadlock) your app if you use it too often.