@Wayne_Miller have you tried polling the socket when you want to see if there is data? Polling the socket will force the DataAvailable event to fire. I’m not saying you necessarily want to write this into your code but for debugging you should be able to use this. Send your command to the other app. Poll the socket - see if you get data. Poll again - see if you get data, etc. If you don’t get anything before the next time you send data to the other app, then you know that it’s not your side but the other side. Also, what happens if you just send a carriage return or some blank command after the first one you sent? Does that get your data to the socket?
TCPSockets are sometimes difficult to understand and work with. Xojo’s documentation on them is not all that great (as can be seen by Greg’s comment about DataAvailable being re-entrant which none of us knew). Nothing anywhere really explains poll. Or how big are the internal send buffers? Can I write commands to the socket prior to connection occurring? If so, how many? Etc. etc. You also have to get yourself into the state of mind of working asynchronously and not serially. That’s the advantage to using an event driven architecture. But it can be difficult to work with and understand and code correctly because you may have a situation where you need to send a specific command, then get the data back and then send something else based on that data, etc. So you need to work serially and wait for specific data while the socket does not. It is not a trivial task!
I do it the other way around (“Pascal” way rather than “C”): when I’m about to send “larger” data (pictures, files, …), I first send a header, which contains the size of the next data. The other end acknowledges and waits for the amount of data (and may time out in case of issues). The first end receives the acknowledgment and sends the data (nothing more, so the other end doesn’t have to split extra data; an acknowledgment will be send from the other end when the expected size is reached).
This has the benefit that I can send whatever data without fearing a delimiter may be inside the data and break the system.
Nice discussion. Thanks. I have done a lot of experimenting with polling: Polling before the first response, polling during the response (and adding the additions), polling after the response has apparently ended to look for empty strings. You made a significant statement that I was leaning towards but not sure about: “If you don’t get anything before the next time you send data to the other app, then you know that it’s not your side but the other side.” I’m now trying to get more information from the folks responsible for “the other side”. I suspect that I am missing something in my protocol (preamble, terminating string) that they are looking for to evoke a response. I suspect that there is something in the subsequent query that evokes a response to the first. Good documentation seems to be a major deficiency in today’s way of doing business. [sigh]
I always use a header as well as a terminating character in my WMABUS products. More importantly, I document well exactly what I’m doing on a character-by-character basis so that, no matter what software development system my client is using, there are no hidden or undefined characters to be supplied by the “system”–great approach and well-defined.
This is not off-topic at all. With all of the great responses I’ve had to my post and the ensuing discussions, I am more and more convinced that the problem is not with Xojo, but with my syntax to the Vector Network Analyzer (you got that right) from Copper Mountain. I have combed their documentation (obviously not carefully enough) and thought I had the data packet correct, but I think (I hope) that I’m missing something. The documentation they supply is copious, so I feel I’m looking for a needle in a haystack. If you are familiar with it, please, give me more off topic information. I fear my entire post may be off topic.
I thought it might be a Copper Mountain VNA. I’m very familiar with the product (my day job is in the RF industry), but not with their protocol specifically. Still feel free to PM me and we can go over the protocol and what you are sending. Happy to help.
This is good! There’s many ways available. I tend to use Greg’s method with something similar to the following:
//Define our DataSeparator
Const DataSeparator = chr(0) + endofline
//See if Data is available, and enough data has sent to obtain a full desired "packet"; also checking to see if more than one "packet" is available for processing.
Var data as String = me.Lookahead
Var tCount as integer = data.CountFields(DataSeparator)
Var termIDX as integer = 0
//Loop thru our available "packets" of "complete data"
For x as integer = 1 to tcount
termIDX = data.IndexOfBytes(termIDX,DataSeparator)
//Only read a complete "data packet"
Var line as String = me.Read(termIDX)
//Separate commands from data to be processed (in the instance of this example)
Var Command as String = line.NthField("::",1).NthField("<{[",2).Trim
//Data does not need to be compressed/decompressed, it merely was in this example to speed up transmissions of data across the network and reduce bandwidth.
Var theData as String = DecompressGZip(line.NthField(command + "::",2))
//Feed the commands and data to a function that appends them to an array to be processed by our timer and popped off the array one-by-one as they're processed
ProcessData( Command, theData )
next
//No actual processing occurs within the DataAvailable event, data is only buffered for processing in this case above.
Interesting, I just put a long running loop inside DataAvailable giving it ample opportunity to yield, fired lots of data at it and I couldn’t get it to re-enter.
As I said above, I wouldn’t count on that. I’d be more worried that it would cause you to lose data because the event couldn’t fire when data came in for the last packet in a stream if there wasn’t going to be any more available later to “push” it through.
So if this is re-entrant and single-threaded when using a timer couldn’t this interrupt the timer and cause the same issues? Xojo is single threaded right or is this handled by an external system-thread?
In my simple test, packets sent over while xojo is “stuck in a loop” are queued at the destination and made available on the next event fire, as expected.
Stacks have an internal queues to cope with this, and they throw errors (like WSAEWOULDBLOCK) when their queue fills if they can’t clear it down quick enough.
In our experience the only moment we lost data is due to the system buffer filled too much. We’ve never lost ANY packet in over > 1m packets, also never had crashes and/or other socket data processing issues (yet). So we’re very curious in why and/or what could go wrong. We use the TCPSocket classes for packet handling in almost real-time with high speed data troughput. So why is this not documented as a precaution that MAY or WILL cause an app to behave other than expected? We use Xojo on all platforms it provides.
So if you’re processing the data in the event handler, and if such processing involves using and modifying socket variables such as pointers and byte counts, then if that is interruptable by another event coming in for the socket, then I fail to see how any processing of the data in the event handler is safe.
In my DataAvailable event handler, all I do is resume the thread that owns the socket (if it’s been paused). That thread then does the ReadAll and should presumably then get all the then-available data.
First of all, I never have had an instance that I know of that I’ve lost data. The data mystically appears after a subsequent query to the test equipment. The query is definitely getting through, but the response is held somewhere that I can’t see. It is not in the receive buffer because I’ve explored that thoroughly using many different approached which a lot of responders on the forum have suggested. There is definitely another buffer that is hidden to me, possibly in the test equipment app, possibly in the Windows OS, or “wherever”.
I should mention that I am using an internal ethernet bus to communicate between Xojo and the equipment app. It uses the native IP 127.0.0.0 in the PC and a port assigned by the equipment app. There is no problem with the Write command, as I can change the parameters of the equipment using Write and see the immediate results on the UI of the app. Where are my responses being held up?
Oh yes, I would agree the the well-placed comment: “I’m confused”
