about the different bufferings on different uart chips :
https://www.youtube.com/watch?v=8YaKgo6-rqI
Actually thats what im saying dont rely on what you get. Your protocol should have a mechanism to know the start and/or end of each packet.
I didnt say the buffer was fixed size.
Its very normal that you dont get what you want in serial comms or even tcp.
A different driver is not going to give you a fixed sized buffer (data).
Dear Aaron,
as Derk has stated, this is not a buffering issue, the DataAvailable event fires at some point the OS decides and presents the information currently in the buffer to you at that time, this does not mean the data being sent is not still there, just that the event fired while it was coming in, if you ReadAll at that point then you are most unlikely to get valid complete data, as you are seemingly expecting.
it is not possible to expect the serial port to know exactly what data you wish to be available in the format you want, it is not psychic.
you must take the data that is there in the event and store it in your own buffer and add the event data as it comes in, that data will need processing and you have to do that in order to meet the protocol you have in the sending device.
you may find the use of LookAhead more fits your needs, it will allow you to look at the data (and not remove it from the buffer), and how many bytes, before you empty the buffer, the buffer is only limited as far as I know by the amount of ram in your system.
in my serial system I use LookAhead to search for the END OF FILE character my protocol uses, and then ReadAll the data out.
the serial port has no idea what your device is sending, it is not sensible to expect what data it has when the DataAvailable event fires is what you expect.
the DataAvailable event is an OS event which has absolutely nothing to do with the data you send as such, its an event that fires when anything is in the buffer at the OS time to set that event happens, I suppose something like ‘its time to fire the serial event, is there anything in the buffer YES>fire event, NO>do nothing’
this is the logic I use to deal with my long standing and perfectly operating serial port based module that is used in every one of my serial port device based applications which are across all the available platforms.
as other people are suggesting, it seems that what you expect is not met in the facts of the serial port operation, however annoying that may be, there is a methodology that must be taken, which is outlined in the posts in this thread, one form of them will be required to be used in order to get reliable data transfer with the serial port, and doing so will get you the result you expect.
Regards
Gosh, I thought the serial port was psychic. That must be the root of all my problems here. Thanks for clearing that up, I really appreciate that.
So it’s not a “buffering” issue, so what? It’s still an issue. The missing bytes aren’t there. They aren’t available in the DataAvailable event, and they aren’t listed in BytesAvailable property. The DataAvailable event fires, I use this code at the start of it:
dim x as string
while me.BytesAvailable > 0
x = x + me.ReadAll
me.Poll
wend
// now work with xSo where are the missing bytes?
[quote=455386:@Mark Carlton]in my serial system I use LookAhead to search for the END OF FILE character my protocol uses, and then ReadAll the data out.
… the DataAvailable event is an OS event which has absolutely nothing to do with the data you send as such, its an event that fires when anything is in the buffer at the OS time to set that event happens, I suppose something like ‘its time to fire the serial event, is there anything in the buffer YES>fire event, NO>do nothing’
this is the logic I use to deal with my long standing and perfectly operating serial port based module that is used in every one of my serial port device based applications which are across all the available platforms.
[/quote]
I’m glad your code is successful. You and others here seem to have missed what I said earlier, namely that I am reading raw bytes from FLASH memory, and there is no signal byte to let me know where the end of the response is, since there doesn’t need to be an end byte – I know how many bytes I have requested to read, and I should get that many bytes back. I’ve said, it doesn’t work reliably for more than 32 bytes at a time, although it is supposed to work with up to 256 bytes at a time, and it did just that before with the FTDI driver, although that driver sometimes caused a bizarre duplicated port, screwing the whole system.
I send a request to read bytes, the request is acknowledged, I send an address, the address is acknowledged, I send a length to read, the length is acknowledged, and then the bytes are supposed to be sent back to me. The chip sends the bytes, but they don’t always all appear in the DataAvailable event.
So I’d like to know how I’m supposed to get the missing bytes when they don’t appear in the DataAvailable event.
Then you should do a while serial.Available >= 32 then
serial.Read(32). To read 32 bytes from the buffer.
Why you are missing bytes could be a bad cable or connection. Or noise on the cable (eg unshielded cheap usb cable).
Als check your baud rate, parity etc
I do not have the FTDI drivers on my system, just the one built in, I have also use the prolific chip set with out issue (using their driver)
without writing the code for you I thought the details in the answers given were clear enough.
the serial port functions perfectly in I don’t know how many hundreds of application around the Xojo world, so if there was an issue with the Xojo class it would have been noticed by now.
IF that is taken as fact, then there must be some other issue, perhaps in your code or perhaps in the hardware you are using.
I personally do not use Poll to check the port so I can offer no experience in that regard, and If I did I am not sure putting it in the DataAvailable event would be where I would try it, also I do not manipulate the code in the event, I call a method and get out of the event as soon as possible, my event code has only a single ‘if’ statement using LookAhead and a call to a method which processes the data, I appreciate that is not how you need to do it, its just an example of it being functional.
you say that with a different driver you get 256 bytes back, did you verify those bytes were correct data or just garbage?
what baud are you using and other parameters?
all this has a bearing on the outcome as you know, try lower baud rates to see if things improve (19200 will likely be 100% sure to work even with very low quality cable over 1 to 2m length), are you sure that the cable is as good as you think, is there outside noise getting in, are you using screened cable, is the transmit hardware working correctly, have you tried a different terminal program to test it?
download ‘CoolTerm’, this is a very good terminal program written in Xojo (not by me) that will show what is possible using the serial port, if it does not work with that application it would suggest there is a hardware problem somewhere.
try connecting the USB serial connector as a loopback, pin 2 and 3 of the 9 pin linked up with a wire, then the terminal will receive all the data you send from it back on its receive window, that tests the USB adaptor.
the biggest issue I have seen with serial ports (of all types) is them being used at far too high a baud rate for the hardware that is in use, I would never try to use anything above 57600 baud on most USB devices as its not reliable over the distance in the cables as they are generally not well screened or of low quality components, they are not a differential port therefore are extremely prone to noise getting in, especially at 3v typically used.
if you want reliable communication over more than a couple of meters, particularly at baud of 57600 and above, use an RS485 converter(at both ends) which is a differential port and can be used reliably, with the correct twisted pair cable such as ethernet, over distances of 1000m and more, not the maximum of 5m that RS232 is capable of (and that at low baud rates).
the RS485 output is no different to RS232 as far as the serial port is concerned, it is just the transport hardware.
beyond those obvious things I am not in a position to offer much more in the way of input.
Yes, I think Xojo is fine and this has to be a driver / OS configuration / hardware issue. I say this because:
- the FTDI driver was working fine apart from sometimes completely screwing up the USB bus.
- FTDI said the Apple driver doesn’t work properly
- the Apple driver seems not to be working properly, like FTDI said
Thank you for all the other information. I had tried all the baud rates already, and slower rates made the missing bytes problem worse. I had checked the bytes received when running the FTDI driver and the data seemed fine. Even when I do get all the data now, it is sometimes corrupted.
So I’ll follow the first advice and test with a different programmer. I’ll have to wait for it to get here, and won’t be doing anything with this until then.
Thank you everyone, for your help.
Could this be an encoding issue ?
I usually specify one, eg: x = x + me.ReadAll( Encodings.ASII )
I have certainly had some issues in the past when I haven’t done so.
It is safer to collect the data from readall into a string first without defining an encoding (leaving it nil). Then read the bytes from the string (i.e. the read buffer) and only after you extract the packets from it, maybe set the encoding on those packets. Otherwise, if you set the encoding to soon, you may have trouble separating the packets out of it.