EndOfLine ASCII 11?

[Christian_Wheel]

This is in a Desktop app. I have a TextArea where a user can paste some text, usually several lines of text.

I need to break this text up into a string array of individual lines, so I use the following code to accommodate the different CR/LF combinations on MacOS and Windows :

var s() as String = TextArea1.Text.ReplaceAll(Chr(13), Chr(10)).ReplaceAll(Chr(10) + Chr(10), Chr(10)).Split(Chr(10))

This eliminates all CRs (ASCII 13) and ensures only LF’s (ASCII 10) at the end of lines on both OSes, and removes empty lines so that each element of the array has a nonempty string.

But a user has come to me with an issue where my code is failing. Somehow their EndOfLine character is Chr(11). I watched them type it and that’s what their keyboard is sending (Their OS is Win11).

Per ASCII charts, 11 is “vertical tab”, and I would have never thought to test for this. I can’t fathom the places that this would break my code over the years in other areas. ASCII 11 isn’t even given an EndOfLine constant in Xojo, so it never occurred to me to test for.

Does anyone know why a user’s keyboard would be using ASCII 11 for EndOfLine? Is this just a thing we need to be testing for all the time when dealing with multiline input? Are there any other EndOfLine characters to know about and test for?

Edit: I should mention that when the user types it, it looks like a regular EndOfLine. I had to write it to a file and examine it in a hex editor to find out what was going wrong.

[Rick_Araujo]

ASCII 11 is not an EOL, but the keycode of that some macOS keyboards send for an alternative RETURN key pressed.

[Christian_Wheel]

So this is something we need to test for all the time if we need individual lines from a textarea? Is this true in web as well?

[Rick_Araujo]

Let me research some code here. Now I’m in doubt if it was 11 or another code.

[TimStreater]

I don’t think keycodes are the same as ASCII values. Where would e.g. function keys fit into that?

[Rick_Araujo]

Wasn’t 11, it’s 3 the alternative code.

[Rick_Araujo]

I don’t know how you got a V-TAB code in your string.

[Christian_Wheel]

After a bit of experimentation, on Windows it seems I can send ASCII 11 to a TextArea by pressing Shift+Return. It is visually indistinguishable from a regular EndOfLine.

Edit: In a non-multiline text field, it shows up as an undrawable character:

[TimStreater]

On my Mac keyboard, RETURN, ENTER, SHIFT-RETURN all produce 0A hex (linefeed). SHIFT-ENTER produces 03 but that is not entered into the textarea.

[Rick_Araujo]

@Christian_Wheel

on Windows it seems I can send ASCII 11 to a TextArea by pressing Shift+Return

I’ve read that in some systems CTRL+K sends this VT too.

[Christian_Wheel]

At least Ctrl+K is a more deliberate keyboard combination.

Shift and Return are adjacent keys and it’d be easy to fat-finger this. Shift+Return is also commonly used in Microsoft Word so if a user has it in their muscle memory it’s much easier to inadvertently use in another app.

If I press Shift+Return in notepad, it inserts a standard windows CRLF and not ASCII 11. I’m unable to replicate this in any non-Xojo app. I wonder if this is a case of the Xojo Textarea being the odd man out, or if this is the behavior of the native textarea on Windows.

[Rick_Araujo]

I’m reading that VT is used to mark a SOFT-RETURN (Shift-ENTER) place in some plaint text editors. So it makes sense.

[Christian_Wheel]

This is easily fixed in my code by changing it to:

var s() as String = TextArea1.Text.ReplaceAll(Chr(13), Chr(10)).ReplaceAll(Chr(11), Chr(10)).ReplaceAll(Chr(10) + Chr(10), Chr(10)).Split(Chr(10))

It just is something that never would have occurred to me so hopefully this thread makes others aware that this can happen.

[Rick_Araujo]

@Christian_Wheel

var s() as String = TextArea1.Text.ReplaceAll(Chr(13), Chr(10)).ReplaceAll(Chr(11), Chr(10)).ReplaceAll(Chr(10) + Chr(10), Chr(10)).Split(Chr(10))

I would

var s() as String = TextArea1.Text.ReplaceAll(&u0d+&u0a, &u0a)_
                                  .ReplaceAll(&u0b, &u0a).Split(&u0a)

[Christian_Wheel]

Out of curiosity (and maybe this is worth its own thread), how do I get the formatting in the code editor to appear as it does in your post? I’d love to horizontally align the .ReplaceAll lines but the Xojo code editor won’t let me indent lines with tabs nor spaces.

[Rick_Araujo]

@Christian_Wheel

how do I get the formatting in the code

I’ve wrote it directly here in the forum, not in Xojo. Like this:

var s() as String = TextArea1.Text.ReplaceAll(&u0d+&u0a, &u0a)_
                                  .ReplaceAll(&u0b, &u0a).Split(&u0a)

Sadly Xojo does not have a free form way of editing with optional manual formatting.

[Beatrix_Willius]

Why aren’t you using any of the EndOfLine constants or even ReplaceLineEndings for this?

[Rick_Araujo]

EndOfLine does not work, VT is not a real EOL, and I’m supposing based on the initial post that ReplaceLineEndings would suffer the same problem BUT I’ve not even tested it.

[Christian_Wheel]

@Beatrix_Willius

Why aren’t you using any of the EndOfLine constants or even ReplaceLineEndings for this?

Old habit to use the ascii codes. But I’m glad I did, if I had been using the EndOfLine constants this would have been trickier to debug without remembering the ascii values. But yeah, as @Rick_Araujo pointed out, ascii 11 doesn’t have a constant and the xojo framework won’t catch it. You apparently need to manually check and compensate for it every time you’re doing EndOfLine detection on Windows.

[Rick_Araujo]

A variant with ReplaceLineEndings:

var s() as String = TextArea1.Text.ReplaceLineEndings(&u0a)_
                                  .ReplaceAll(&u0b, &u0a).Split(&u0a)