Funny characters while reading or writing PString in TextArea

In my app, I read and/or write PStrings in a TextArea (not TextField) but I do not get a carriage return, i.e. the text does not go into a new line, instead I get a sequence of characters (something like :black_medium_square::black_medium_square::black_medium_square:) right along the text.
Is there something I’m missing?

Thank you

Very likely an Encodings issue. Where is the string sourced. Be sure to DefineEncoding on the string, when read.

Do you mean also encoding before writing as WritePString?

To be sure: yes.

But, why using PStrings ?

You do get a String from your TextArea? Or I am mistaken there?
You would need to convert the encoding on the string before you convert the string to a PString.

You are aware that PString is limited to 255 characters (actually bytes, I believe. I don’t know what changes Apple introduced in PString, but PString actually comes from Pascal String. When Niklaus Wirth, the designer of Pascal created this type of strings, he had extended ASCII in mind. No Unicode, no double byte character encodings).

Those black squares are not helpful. Could you please give us hex bytes? Best a short text from the TextArea with the corresponding bytes in the stream.

Honestly, I don’t know. I tried to do the normal way (not Pascal string) and, for some reason it failed so I stayed on

Gladly but, please, tell me how to get the bytes from the stream. I’m a little green on reading and writing files

Write the stream to a file and use a hex editor, hex viewer or use hexdump from a terminal (if you are on Linux and likely also on Mac)
As an alternative you can create a stream to a MemoryBlock and inspect it’s content in Xojo’s debugger.

Var mb As New MemoryBlock(0)
Var bs As New BinaryStream(mb)
bs.WritePString(MyPString)
bs.Close

(This code example is adapted from a code sample in Documentation for BinaryStream.Close. Don’t know if this actually works, as writing to a 0-sized MemoryBlock is not documented elsewhere.)

That’s valid code. The MemoryBlock will be automatically resized as you write to the BinarySyream.

You’re writing the TextArea to a file using BinaryStream.WritePString, then reading that back using BinaryStream.ReadPString, am I correctly understanding? Or is it rather an internal PString in your code?
When a string isn’t binary data (i.e. text), setting the encoding is usually mandatory. The TextArea normally has a default encoding, so you must make sure to set the same encoding when you read the string back.

I always extend the BinaryStream class to handle longer “PStrings”, because of that limitation:

Public Sub WriteLongPString(Extends b As BinaryStream, Text As String)
  b.WriteInt32 Text.Bytes
  b.Write Text
End Sub
Public Function ReadLongPString(Extends b As BinaryStream, Encode As Boolean) As String
  var e As TextEncoding
  
  if Encode then e=Encodings.UTF8
  
  Return b.Read(b.ReadInt32,e)
End Function

Another way is to directly use Xojo. When the string is read back (again, assuming we’re talking about BinaryStreams), inspect it in the variables viewer:

I made it.
Thanks everybody for your help

Wouldn’t you want to share your solution so future users with the same issue don’t ask again?
(and so we can also tell you whether you did it the best way)

I will as soon as I’m back on the Mac. Right now I am out using the iPhone

Here I am.
The way I did it is:
in writing saving the tect in the areaText I wrote BiSt.WritePString(M.Note.ConvertEncoding(Encodings.UTF8))

in reading I wrote M.Note = BiSt.ReadPString(Encodings.UTF8)

It’s probably not the correct way to do it but it works (at least in my app)

ConvertEncodings does that it changes the text from one encoding to another. If you have declared the source string is UTF8 then using ConvertEncoding to convert to UTF8 will do exactly nothing, as it already thinks that it is UTF8.

Strings will often have a nil encoding, which is Xojo’s way of saying it doesn’t know what the encoding is. In that case it will still not do anything when ConvertEncoding is called as it doesn’t know how to complete the task.

In order for it to work it needs to be the correct encoding and then you can convert it to what you require, which for the interface is generally best as UTF8.

You can use DefineEncoding to tell Xojo that the string is using a given encoding and then ConvertEncoding if required.

All of what you said, plus the clarification that if you have a string with a known encoding, and you convert to another encoding, the bytes in the string will be altered. IOW, it’s not a just a bit of metadata about the String that is changed. DefineEncoding, however, does not alter the bytes, just the metadata.

1 Like

Let me understand: are you sayng I can do without ConvertEncoding in either reading or writing?

If you are seeing strange squares then it is very likely that your data is not UTF8. If it was the string would display correctly. You need to figure out the encoding being provided and use DefineEncoding once read and ConvertEncoding to display it.

Once again if you are altering it and returning it to the source you would need to ConvertEncoding back to the format your external source is expecting before passing it back to the external source.