Interpreting &h00 as null termination with SerialConnection

General
1

Hello –

I’m trying to write and read binary data between Xojo and an embedded system via UART using SerialConnection. I want to be able to send and receive arbitrary data. This includes using values of &h00 without it being interpreted as a null terminator on either end. On the embedded system, it’s no problem to receive something when sent like this from Xojo:

Var mb As New MemoryBlock(10)
mb.Byte(0) = &h0a 
mb.Byte(1) = &h05 
mb.Byte(2) = &h01 
mb.Byte(3) = &h04 
mb.Byte(4) = &h12 
mb.Byte(5) = &h34 
mb.Byte(6) = &h00 <<<--- This is the problem.
mb.Byte(7) = &h00 
mb.Byte(8) = &h0D
mb.Byte(9) = &h0A
Win_Main.SerialConnection1.Write(mb)

It’s no problem to receive the data on the embedded side. From the embedded side, I’ll copy the above received data into the embedded transmit buffer and send it right back to Xojo. When doing this, SerialConnection.LookAhead will stop at mb.Byte(6), which is &h00. This makes me think that Xojo is interpreting the &h00 byte as a null terminator such that it thinks it’s the end of the serial transmission and it (Xojo) stops reading data.

Within my embedded system, I can modify mb.Byte(6) and mb.Byte(7) to be non-zero. I happened to set them to values like 0x6 and 0x7, but I don’t think it matters, so long as they are not &h00. Xojo receives the message correctly all the way through mb.Byte(9). I can observe all of this in the Xojo debugger when read like this:

	Var data As String
	// Read the buffer (non-destructively) to see what's there
	data = Me.LookAhead(Encodings.ASCII)

Using

data = Me.LookAhead(Encodings.UTF8)

makes no difference (no surprise). Noob question, I’m sure, but how can I actually receive binary data on the Xojo side without interpreting 0x00 as a null terminator/end of string character?

Thanks for and potential thoughts and suggestions.

2

If you want raw data, don’t specify an encoding. A String with no encoding is just a bunch of bytes. In which case, &h00 is a valid byte value. In any encoding, it is not.

3

Thanks for your reply, Tim. I tried that, as this:

Var data As String
Var mb As MemoryBlock

// Read the buffer (non-destructively) to see what's there
data = Me.LookAhead()
mb = Me.LookAhead()

and it still failed. Both data and mb show the same thing – only 5 bytes where there should be 14, and matching data. Did I make a mistake in my code?

Also, if I increment all bytes in the message by 1 before sending from embedded to Xojo, it works as expected, and all 14 bytes are recognized.

Thanks.

4

You should wait for more data and add the data to your own buffer.

DataAvaioable has NO guarantees it will receive all data.

Read your data to a property on an subclass or window etc.

Example:

Buffer as string

In dataAvailable:

Buffer = Buffer + Me.ReadAll(Nil)

5

The problem is that you’re using a String for the buffer. All of the output mechanisms in Xojo assume that &h00 is a null terminator, so even if the string contains data beyond the null, it won’t be shown.

Instead, try using a memoryblock.

6

Hi there, and thank you for your suggestions!

I’ve had another idea that I’m going to look into first – perhaps it’s not that Xojo isn’t interpreting things correctly, but rather that my embedded side DID NOT SEND the data for similar reasons – that it interpreted the 0x00 as a null termination. I’m going to look into that this morning.

Still, regarding the two most recent responses…

DerkJ – Good idea, but I’m sure all the data is there. I’m stepping through code by hand using a debugger on both sides, and I see it executing and sending. There’s no race condition.

Greg O – I’m doing it both ways already and I get the same thing when using a String or MemoryBlock. But please let me know if you see that I’ve made a mistake. I did it like this:

Var data As String
Var mb As MemoryBlock

// Read the buffer (non-destructively) to see what's there
data = Me.LookAhead()
mb = Me.LookAhead()

Thanks again!

7

Just out of curiosity, when you finish writing on the sending side, did you call Flush on that socket? It may be that the data is just waiting.

8

I’m not sure there’s a flush to call on the embedded side; it may be all in HW. I don’t own the whole stack, for better or worse. However, in looking at the code, I’m 99% sure that it’s on the embedded side and that it is the one interpreting the 0x00 as a null such that it stops transmitting there. I need to make some changes and test further, but I’ll let you know. It may be that Tim’s initial response is the right one.

Thanks.

9

Yes, it was the embedded side ending transmission based on its observation of the null char (0x00). I changed the code there to transmit a count rather than watching what’s being sent and it works fine on the Xojo side. Indeed, both of these result in the same thing, so I didn’t even really need to get rid of the encoding specification.

Var data As String
Var mb As MemoryBlock

// Read the buffer (non-destructively) to see what's there
data = Me.LookAhead(Encodings.ASCII)
mb = Me.LookAhead()

My apologies for posting a failure that was (kind of) my own doing, but I did learn something.

Also, this being my first post, I really appreciate the quick and helpful responses. Thank you!

1 Like