Character "°" causes crash in web IDE debugger

Targets Web
1

I have data stored in a MySQL database that includes the degrees symbol in a specific column. (Typically as “D50/2°”) I query the data into a recordset and then store it in a two dimensional array to then display it in a listbox. If I try to reference the data for that specific column in the array in any way including s = ReplaceAll(ScanData(i,n),"°","degrees") while I am looping through the array to populate the listbox, the IDE debugger crashes loading the app page. If I skip over the data and leave that column in the listbox blank it runs fine. I also tried a replaceall from the recordset before storing in the array but get the same crash. Anyone have any other ideas for a workaround? The character works just fine in a desktop app.

Version 2014 R 3.2

2

Please file a bug report

3

[quote=171896:@Tom Dixon]I have data stored in a MySQL database that includes the degrees symbol in a specific column. (Typically as “D50/2°”) I query the data into a recordset and then store it in a two dimensional array to then display it in a listbox. If I try to reference the data for that specific column in the array in any way including s = ReplaceAll(ScanData(i,n),"°","degrees") while I am looping through the array to populate the listbox, the IDE debugger crashes loading the app page. If I skip over the data and leave that column in the listbox blank it runs fine. I also tried a replaceall from the recordset before storing in the array but get the same crash. Anyone have any other ideas for a workaround? The character works just fine in a desktop app.

Version 2014 R 3.2[/quote]

You may want to look closely at the data, byte by byte. Such crashes occur frequently because a control character is hidden there. So the culprit would be that control character and not the degree.

4

And make sure you define the encoding when you pull the string out of the database.

5

OK. More Info. The character maps as ASC 176. There is no other control character in that database field. The default character set of the database is utf8. I set the encoding when retrieving the data from all fields to utf8 and it still crashes.

s = ConvertEncoding(rs.IdxField(n+1).StringValue, Encodings.UTF8) ScanData(i,n) = s

The browser error dialog when the app crashes shows an unknown character"?" instead of the degree character. \ Xojo.controls['VE9YnS04'].setCellContents(0,11,\"D50\\/2?")

This doesn’t work any better:

s = ReplaceAll(ConvertEncoding(rs.IdxField(n +1).StringValue, Encodings.UTF8),"°","degrees") ScanData(i,n) = s
and also:

s = ReplaceAll(ConvertEncoding(rs.IdxField(n +1).StringValue, Encodings.UTF8),Chr(176),"degrees")

but I still get

\ Xojo.controls['Vq8ycraD'].setCellContents(0,11,\"D50\\/2?")

So apparently the replacement isn’t working.

So I tried this:

For i = 1 To Len(s) s = ScanData(0,13) t = Mid(s,i,1) MsgBox Str(Asc(t)) Next

Which gives me 68, 53, 48, 47, 50 and 176 or “D50/2°”

So I’m stumped on what to do next. I can’t seem to be able to replace the character which is weird and why it is unrecognized is also weird.

6

[quote=172038:@Tom Dixon]OK. More Info. The character maps as ASC 176. There is no other control character in that database field. The default character set of the database is utf8. I set the encoding when retrieving the data from all fields to utf8 and it still crashes.

  s = ConvertEncoding(rs.IdxField(n+1).StringValue, Encodings.UTF8)
  ScanData(i,n) = s[/code]

[/quote]

This is the incorrect way to do this
When you get data from the database it has an undefined encoding in the recordset
Its a bunch of bytes
Using convert encoding on “a bunch of bytes” results in “a bunch of bytes” still without a defined encoding

Try this instead
s = DefineEncoding(rs.IdxField(n+1).StringValue, Encodings.UTF8)

7

Norman that still gives the same error. I even tried:

s = DefineEncoding(rs.IdxField(n+1).StringValue, Encodings.UTF8) t = ReplaceAll(s,Chr(176),"") ScanData(i,n) = t

The character doesn’t get replaced and I still get

\ Xojo.controls['Yaky1tGn'].setCellContents(90,11,\"D50\\/2?")

8

However just on a whim I tried:

s = DefineEncoding(rs.IdxField(n+1).StringValue, Encodings.WindowsANSI) ScanData(i,n) = s

And that worked

9

[quote=172052:@Tom Dixon]Norman that still gives the same error. I even tried:

s = DefineEncoding(rs.IdxField(n+1).StringValue, Encodings.UTF8) t = ReplaceAll(s,Chr(176),"") ScanData(i,n) = t

The character doesn’t get replaced and I still get

\ Xojo.controls['Yaky1tGn'].setCellContents(90,11,\"D50\\/2?")[/quote]
That can happen IF your data ISN’T UTF-8

Since you’re using mysql you CAN force it to give you utf-8 data if you issue the command
db.SQLExecute( “set names ‘utf-8’” )
when you connect to it

https://forum.xojo.com/13897-encoding-troubles-with-2014r2

10

The DB Character set when it was created was utf8 so why it’s not is a mystery but knowing I can force it to give me utf8 is acceptable. I’ll experiment with that and see what happens.

11

Check what your severs encoding is defaulted to
You can put IN any encoding and the database may convert it to whatever it was configured to use
And you can tell the database to pull it out and return it in a different encoding

Regardless, the plugin returns “bytes” so you need to use define encoding on returned string values
And if you try to define a bunch of bytes as an encoding that those bytes are invalid in then the encoding of the string will be wrong
So you need to be 100% SURE the plugin is giving you bytes that ARE UTF-8 IF you’re going to define the encoding to be UTF-8

That mysql command WILL tell mySQL to return the correct UTF-8 bytes for the data you’re grabbing

Basically you need to do that otherwise you aren’t sure what you’re getting and then problems ensue

12

That isn’t UTF-8. It’s probably either Latin1 or MacRoman. You HAVE to know the encoding of the data and Define it before you use it.

13

Now I understand much better why the new framework forces to explicitly set the encoding in such a case.

14

+10

15

ASC 176 is the degree symbol in Latin1. It is ASC 161 in MacRoman. In UTF-8, it is codepoint &uB0 (ASC 176) and is encoded in the string as &hC2B0. I bet your string is actually Latin1. Try

s = DefineEncoding(rs.IdxField(n+1).StringValue, Encodings.ISOLatin1)