Arduino Serial Comm Unreliable

General
1

I have an Arduino that I have programmed to emulate a Pinewood Derby timer.

When I have the “serial monitor” built into the Arduino IDE running it receives the data just fine, every time.

But when I try to receive the same data in my Xojo project the data gets mangled. More specifically, the data received in Xojo is not full lines that end at the LF CR terminators.

When I have the Arduino Serial Monitor running I see the Arduino sending:

A=3.307 B=3.749 C=2.573 D=4.158 E=0.000 F=0.000
A=3.430 B=3.772 C=4.044 D=3.378 E=0.000 F=0.000
A=4.423 B=4.209 C=2.940 D=2.665 E=0.000 F=0.000
A=2.992 B=3.542 C=4.487 D=3.003 E=0.000 F=0.000

And so it continues

Keep in mind that this data is coming through the USB cable, through the serial port named USBModem1411, and being displayed by the Arduino Serial Monitor program.

Now when I try to receive the same data through a serial connection inside Xojo I get:

A=3.307 B=3.
749 C=2.573 D=4.158 E=0.000 F=0.000@@A=3.430 B=3.
B=4.209 C=2.940 D=2.665 E=0.000 F=0.000@@
A=2.992 B=3.542 C=4.487 D=3.003 E=0.000

So you see I’m getting the data … but it’s getting chopped up and fed to my program in irregular, chopped up chunks instead of a full line at a time. Where I’m showing the @ symbols the actual screen output has a black diamond with a ? inside. I’m guessing those are the CR and LF characters that are supposed to terminate the lines.

I have a Serial gadget attached to Window1.

I have a window with a drop down list named “PortSelect” where I have the available serial ports listed and in the DropDown’s “Change” event I have:

If PortSelect.ListIndex = 0 then
Window1.Serial1.close
PortOpen = false
else
Window1.Serial1.close
i = PortSelect.ListIndex - 1
Window1.Serial1.SerialPort = System.SerialPort(i)
Window1.Serial1.Baud = Window1.Serial1.Baud9600
Window1.Serial1.Bits = Serial.Bits8
Window1.Serial1.Parity = Serial.ParityNone
Window1.Serial1.Stop = Serial.StopBits1
Window1.Serial1.CTS = False
Window1.Serial1.DTR = False
Window1.Serial1.XON = False

PortOpen = Window1.Serial1.Open

if PortOpen = false then 
  MsgBox "The port is not available"
  exit
end

In the Serial1.DataAvailable event I have:

Dim TempStr As String
if PortOpen = true then
Select Case LaneCount ’ read each lane’s time

Case 1
  TempStr = Serial1.ReadAll                   'read the serial data
  SerialMon.TextField1.text = TempStr    'show it to me
  'Case 2
  'TempStr = Serial1.ReadAll
  'SerialMon.SerialInput.AddRow(TempStr)
  'Case 3
  'TempStr = Serial1.ReadAll
  'SerialMon.SerialInput.AddRow(TempStr)
  'Case 4
  'TempStr = Serial1.ReadAll
  'SerialMon.SerialInput.AddRow(TempStr)
  
End

LaneCount = LaneCount + 1
if LaneCount > NumOfLanes then
  LaneCount = 1
end

end

Don’t worry about the LaneCount part because I was going to use that to read each lane’s time, but for now I’m just trying to get the data received to show up correctly in a single line and I’ll worry about parsing the data later.

As you can see above I’m reading the data with “Serial1.ReadAll” and then sending that text to a TextField to be viewed. I have also tried “Serial1.Read(256)” and “Serial1.Read(30)” with the same result of the lines being chopped up and not delineated at the line terminator characters as they should be.

I have pulled out just about the last hair that I have on my head trying to figure out why I can read the data perfectly in the Arduino Serial Monitor program, but Xojo serial input keeps mangling the data so badly.

2

I forgot to mention that I’m doing this on a Mac running OSX 10.9 (Mavericks) and I’m compiling as a Cocoa app

3

Marc you need to verify you’re receiving a suitable data packet. The LookAhead method allows you to view the buffer to determine if you have enough data to process. You’ve mentioned above that there is a some kind of line ending so check for that before processing the data and don’t use readall.

Suppose the Line ending is EndOfLine then you can add at the top of your dataavaialable event

If InstrB(0, me.lookahead, EndOfLine) = 0 Then Return - you don’t want to process yet because there isn’t a record to process.

Then you should process the data in a loop looking for the endofline/endofrecord and only call the data that is complete. I use

Dim i As Integer = InstrB(0, me.LookAhead, EndOfLine)

Do Until i = 0
Dim s As String = me.Read(i - 1)
Call me.Read(Len(EndOfLine) ’ Drop the EOL chars
// Process s
i = InstrB(0, me.lookahead, endofline)
Loop

The whole thing about event driven processing is that you have NO control over when the event’s fire, so YOU have to validate the data.

HTH

Wayne

4

The following thread provides three examples of a simple terminal. The first one only uses the DataAvailable event and sometimes “misses” stuff. The second and third one use a timer to either “poll” (Serial.Poll()) the port for data (triggering the DataAvailable event), or perform the reading from the port directly in the timer until the buffer is empty (bypassing the DataAvailable).

https://forum.xojo.com/12492-need-a-simple-serial-terminal/34/

https://dl.dropboxusercontent.com/u/42062295/RealSoftware/forum%20serial%20monitor.rbp
https://dl.dropboxusercontent.com/u/42062295/RealSoftware/forum%20serial%20monitor_2.rbp
https://dl.dropboxusercontent.com/u/42062295/RealSoftware/forum%20serial%20monitor_3.rbp

5

Thankyou @Wayne Golding

You learn something new every day! your method sounds really good!
At present, i do a Readall into a buffer, then process the buffer, which works ok, but i always felt it wasn’t the most elegant
solution.

6

Thank you for your fresh perspectives on the problem. After reading your posts I realized that the problem was that my program was reading the serial buffer faster than the Arduino could send, which caused my program to only grab ‘what was available’ each time the “data.available” event instead of waiting for the ‘data send’ to be completed.

I tried several of your suggestions with limited success. That is, I was getting more of the data, but still getting incomplete transmissions.

But the suggestion of a Timer led me to a successful solution.

In the Serial gadget’s ‘Data.Available’ event I enabled a timer that had a time of 100ms, which was about 3 times longer than the longest data stream I am expecting. So when the data input triggered the ‘Data.Available’ event, then it would start the timer in ‘single’ mode. Then when the timer timed out the full data transmission would be in the serial buffer, and within the Timer’s Action event I have it perform a Serial.ReadAll() to retrieve the full contents of serial buffer … followed by a Timer.ModeOff to rest the timer in preparation for the next data send.

After I have the data in a string variable, it is a simple matter to strip off the line terminator characters and proceed with parsing the data that has been received.

Thank you for your help and suggestions.