Inconsistency reading data from serial port

Hi all,

I’ve been all over the place with this one, trying numerous different angles and I still can’t figure out where I’m going wrong.

I have a “dongle reader” and some “dongles”:

The reader hooks up via a proper serial port (not a USB-Serial adapter, not a Bluetooth-Serial adapter… a proper 9-pin RS232 port).

When I run a serial terminal app (Termite in the screenshot) I consistently receive 15 bytes every time I touch one of the dongles to the reader:

Now, I would like to read this data programmatically myself so I can actually make use of it.

So, I create a simple application with a single TextArea and a SerialPort. Here’s my code:

[code]Window1 -> Controls -> Serial1 -> DataAvailable:
DIM DongleID as MemoryBlock = Me.ReadAll()
DIM Size as Integer
Size = DongleID.size
TextArea1.Text = TextArea1.Text + “Read " + str(size) + " bytes.” + EndOfLine

Window1 -> Event Handlers -> Open:
Serial1.SerialPort = System.SerialPort(0)
IF Serial1.Open Then
RETURN
ELSE
MsgBox “Error opening the serial port.”
END IF[/code]

When I run it, I get the following output:

As far as I can tell, there aren’t any end-of-line terminators that I can check for to determine if the string is complete or not:

There doesn’t appear to be any reason why it would sometimes read 15 bytes and sometimes read 7 followed by another 8, but I would really like to be able to read 15 bytes consistently.

Any thoughts?

You simply can’t be sure that you have all the data you’re looking for when the event fires when dealing with serial ports. You need to build up a new packet of data on seeing a start byte and send it for processing only once an end byte has been received.

What are the [0F] and [1D] that seem to wrap some data? Could they not be taken as delimiters for the purpose of checking the data is complete?

The [0F] and [1D] are bytes that can’t be represented in ASCII.

In the first Termite screenshot it shows me pressing 2 different dongles on the reader, twice each.
In the second Termite screenshot it shows my pressing a single dongle on the reader three times.

It seems that the structure of the data being sent is 9 bytes that never change (at least they don’t across the 6 dongles I currently have) followed by 6 bytes that is a unique identifier for the specific dongle. The [1D] is therefore part of the dongle identifier and can’t reliably be used as a delimiter. You’ll notice the [0F] corresponds with the first byte (0F) read in each instance in the first Termite screenshot.

My problem is that sometimes 15 bytes are read/sent when I press the dongle, and sometimes it is as if 2 separate send events are triggered, one with 7 bytes and then immediately another (without an additional dongle press) with 8 bytes… and I have no way of knowing when I’m going to read 15 or when I’m going to read 7 or 8.

Would it be worth attempting this:

If size = 15 bytes then Do something with the data Else If size = 7 bytes then Store in a global variable Else If size = 8 bytes then Add to the global variable Read in all 15 bytes from the global variable Act on the 15 bytes End If

Or do you think, since it’s the last 6 bytes are all that I’m interested in, this would be better:

If size = 15 bytes then Read the last 6 bytes and act on the data Else If Size = 8 bytes then Read the last 6 bytes and act on the data Else Ignore the data since it won't contain the dongle code anyway End If

Or am I barking up the wrong tree and there’s a much simpler way to sort this problem?

I went with the latter of my proposed options and it works. It’s a bit of a hack, but it works.

If size = 15 or size = 7 then Extract the last 6 bytes of the packet and do something with it End If

Please note that many terminal programs (especially the ones written with a programming language that directly accesses the COM port physical layer) can receive data in pretty much real time. Xojo does not subscribe to that model of access (and that is not necessarily a bad thing - you just have to know how to handle this), so its data reception is event-based and can be delayed or broken up in chunks.

Just keep in mind that while your “hack” may work on your PC, it may not on others (depending on the system, how many applications are vying for processing power, etc); unless you can be sure that at the hardware level the information really come in two packets (i.e. 7 and 8 bytes) which sometimes arrive together and sometimes do not. In other words if the information from the dongles only comes in one packet, you could easily have a system that sees all 15-bytes at once, 7-bytes on the first event and 8-bytes on the next one, 10-bytes one the first event and 5-bytes on the second one, etc. Normally serial dongles include a common 1st (and maybe 2nd) byte which can use to indicate the beginning of the packet, and a common last (and maybe next-to-last) byte that indicate the end of the packet. IMHO you should use the Data Available event to “assemble” the packet, and use a timer to check the status of the “assembled” packet so when it is ready you can make use of it.

Just saying, those are not dongles but fobs. Or RFID fobs/tags.
In context; Originally, a dongle is something you plug into your machine (and leave it there) to authenticate/unlock software so you can start your Autocad or run your accounting software.
A fob is something you attach to your keyring, hold it in front/against a reader to open the door etc.

Are your serial port settings correct? (9600, 8 bits, no parity, 1 stop bit) If not correct but there’s still a handshake, you get inconsistent data.
Try TextArea1.Text=TextArea1.Text+Me.ReadAll() to see what’s actually coming in.
Also see if switching on DTR makes a difference. Or maybe even CTS. If the DSR supports that, both will make sure they’re ready to receive data.

Why don’t you just leave the data in the buffer until you have 15 bytes? Something like this in the data available event

[code]If LenB(me.lookahead) < 15 Then return

Data = me.read(15)
[/code]

Should do the trick.

I’d be willing to bet that 0F and 1D are the delimiters to the packet.

or that 0F is “number of bytes in whole packet” which is 15 :stuck_out_tongue:

Sure seems that way (0x0F and 0x1D). Curious (or not) that those are defined as non-printable ASCII characters (0x0F → Shift In, 0x1D → Record Separator).

[quote=@Marco Hof]Just saying, those are not dongles but fobs. Or RFID fobs/tags.[/quote] Whatever you may want to call them (and yes they look like FOBs) the OP called them dongles, so dongles it is. The point is that unless you know how to delimit the message content, the message will be next to unusable.

[quote=@Wayne Golding]Why don’t you just leave the data in the buffer until you have 15 bytes?[/quote] You can read as many 15-byte data packets as you want, but if you don’t know where they start, they are next to useless. You need to somehow synch the packet (i.e. know where it starts, and also hopefully where it ends) and based on the data shown it seems all the needed information is right there.

[quote=@Grant Phillips-Sewell]There doesn’t appear to be any reason why it would sometimes read 15 bytes and sometimes read 7 followed by another 8, but I would really like to be able to read 15 bytes consistently[/quote] I think that is now explained above. One way to get the data is to keep checking on the DataAvailable event for the start of the package and then acquire the next 15 bytes (and perhaps check the end of packet delimiter).

[quote=234564:@Marco Hof]Just saying, those are not dongles but fobs.[/quote]Fob then. Since I have explained how the thing works and provided an image, I would have thought that people would understand what type of device it is.[quote=234564:@Marco Hof]Are your serial port settings correct? (9600, 8 bits, no parity, 1 stop bit) If not correct but there’s still a handshake, you get inconsistent data.[/quote]No idea. I have no access to the software that supposedly originally came with the fob reader, but the defaults of 9600,8,0,1 seem to work perfectly well in Termite (as per the screenshots).[quote=234564:@Marco Hof]Try TextArea1.Text=TextArea1.Text+Me.ReadAll() to see what’s actually coming in.[/quote]What’s actually coming in is a series of numbers, letters and a few non-printable characters. Indeed, the 6 fobs I have give the following data (in hex):

0F 49 30 30 30 30 31 33 34 39 38 46 45 42 1D 0F 49 30 30 30 30 31 33 34 39 39 38 36 41 88 0F 49 30 30 30 30 31 33 34 39 31 41 45 32 0C 0F 49 30 30 30 30 31 33 34 39 32 38 30 46 35 0F 49 30 30 30 30 31 33 34 41 35 39 35 46 B4 0F 49 30 30 30 30 31 33 34 38 39 43 45 31 70[quote=234564:@Marco Hof]Also see if switching on DTR makes a difference. Or maybe even CTS. If the DSR supports that, both will make sure they’re ready to receive data.[/quote]They made no positive difference.[quote=234593:@Tim Hare]I’d be willing to bet that 0F and 1D are the delimiters to the packet.[/quote]As you can see from the data above from the 6 fobs, 0F is present at the beginning of each reading, so is likely to be an indicator of the length of the packet, but 1D is only found in a single instance (this can also be seen in the first Termite screenshot - these entries ended in 35 and B4) so I really don’t think 1D represents the end of the packet.[quote=234571:@Wayne Golding]Why don’t you just leave the data in the buffer until you have 15 bytes? Something like this in the data available event[code]If LenB(me.lookahead) < 15 Then return

Data = me.read(15)[/code]Should do the trick.[/quote]I hadn’t tried this because I hadn’t thought about it. To my mind the “DataAvailable” is event triggered so each time data is sent (or is found to be readable) on the serial line, this event gets run… so if there’s not a full packet when the event is triggered, then you’ll only get a partial packet again when the event is triggered again. Are you saying that simply using lookahead I can say “the event has been triggered, but I’m not going to read from the buffer yet because the info in the buffer doesn’t appear to be complete yet”? If so, then this may well be the answer.

That is correct.

I wonder if that last character in the packet isn’t some kind of checksum.

[quote=234831:@Tim Hare]That is correct.[/quote]Excellent! I’ve never done much work with serial devices before (besides hooking up Cisco kit to configure it over an RS232 port). That makes my life a lot easier - wait until I have the right amount of data, then boom work on it when it’s all there.

Thank you all. :slight_smile:

I’m really new in xojo but my interest is using serial ports and sockets , so this lead me to try example for serial port , and found that
reading (barcode scanner) is not accurate, which means that first cca. 5 readings is good ,then begin to receive broken string. Receiving 0012345 instead 001234567895 and you never know will next read is good or false,if I have right same problem like
Grant with reading RFID tags.Hope that is not bug in xojo serial driver and that solution exists!