Emojis and JSON

Getting Started
1

I´m getting my feeds wet with the Facebook API and have run in to a problem.

In a web app I have this:

The JSON data is coming in to “TextArea1” with this code in the action event of a button:

  Dim s As String
  Dim r As String
  Dim https As New HTTPSecureSocket
  
  s = EncodeURLComponent(FaceBookURLField.Text)
  r = https.Get("https://graph.facebook.com/v2.7/" + s + "/posts?limit=" + FaceBookNumberField.Text + "&" + FaceBookAccessToken, 30)
  
  r = DefineEncoding(r, Encodings.UTF8)
  
  TextArea1.Text = r

“FaceBookURLField” is a Text Field where I put data like “goxojo” to get a Facebook page.
“FaceBookNumberField” is a Text Field to limit the number of results.
“FacebookAccesToken” is the token for my Facebook App.

The JSON data in “TextArea1” validates fine on services like: https://jsonformatter.curiousconcept.com

BUT i get a RunTimeException (“The data could not be converted to text with this encoding.”) using the next piece of code to populate “Listbox1” when there are Emojis like “FACE SCREAMING IN FEAR” ("\ud83d\ude31") in the JSON data:

[code] Dim s As String = TextArea1.Text
Dim js As New JSONItem(s)
Dim results As JSONItem = js.Child(“data”)
Dim n As JSONItem

ListBox1.DeleteAllRows
For i As Integer = 0 to results.Count - 1
n = results.Child(i)
if n.HasName(“message”) Then
Listbox1.AddRow(n.value(“message”).StringValue)
End if
Next
[/code]

What am I doing wrong?

Best regards
ojan

2

Offhand, I’d say it’s not you, it’s JSONItem and lack of handling of UTF-16 surrogate pairs. See:

<https://xojo.com/issue/35010>

The solution is to switch to the new framework handling of JSON. You’ll have to change your code a bit, but it will be minor.

3

Or you can use my JSONItem_MTC class, a drop-in replacement for JSONItem that addresses some of its shortcomings. You can find that at:

https://github.com/ktekinay/jsonitem_mtc

But at this point, I recommend the new framework.

4

Can you paste the actual JSON result you get

5

@Kem Tekinay (Irony alert) Thanks for nothing. Why didn’t you just tell me that I put some freaking parentheses wrong. Now I have to learn a lot more. AAAARGGGGH!!!
Honestly. Thank you very much for you time. I’ve already studied your “JSONItem_MTC class” but fumbling around the edges of my programming skills I dared not trying.

@Norman Palardy Sure, and thank you too :slight_smile:
The “goxojo” does not generate errors right now so I tried with https://facebook.com/bbcnews and It generates the error right now

{"data":[{"message":"We are #LIVE at a rally of Donald Trump supporters near the Republican National Convention in Ohio. The BBC's Rajini Vaidyanathan is asking why are they voting for him.","story":"BBC News is live now.","created_time":"2016-07-18T19:36:40+0000","id":"228735667216_10153763838212217"},{"message":"Dani Mathers posted it saying \"If I can't unsee this then you can't either\".","created_time":"2016-07-18T19:34:38+0000","id":"228735667216_10153763880042217"},{"message":"A good match for Harrison Ford's character?","created_time":"2016-07-18T19:02:47+0000","id":"228735667216_10153763819807217"},{"message":"The man identified as the killer of three US police officers had posted videos complaining at police treatment of African Americans and urging them \"You gotta fight back.\"","created_time":"2016-07-18T18:29:40+0000","id":"228735667216_10153763748282217"},{"message":"Two private broadcasters say the general had denied playing a role. But the state-run Anadolu agency say he told interrogators he \"acted with intention to stage a coup.\"","created_time":"2016-07-18T17:56:43+0000","id":"228735667216_10153763671317217"},{"message":"Politicians are debating whether or not the UK should renew the Trident nuclear weapons programme. They are due to vote later.","created_time":"2016-07-18T17:27:48+0000","id":"228735667216_10153763594467217"},{"message":"The video of #EricGarner's arrest two years ago has since inspired a network of \"copwatchers\" - ordinary people who film the NYPD in action. Documentary maker Ben Steele has followed some of the New Yorkers who film the police.","created_time":"2016-07-18T17:27:00+0000","id":"228735667216_10153763068177217"},{"message":"The two-day window for people to register as a supporter of The Labour Party is now open. Read our full guide: http:\\/\\/bbc.in\\/29Iv7tY","created_time":"2016-07-18T17:04:19+0000","id":"228735667216_10153763542787217"},{"message":"Will Britain keep its nuclear deterrent? As MPs prepare to vote on whether to renew #Trident, Catriona Renton was at Faslane, Scotland where the nuclear-armed submarines operate from. She took your questions. #FacebookLive","story":"BBC News was live.","created_time":"2016-07-18T16:22:00+0000","id":"228735667216_10153763450982217"},{"message":"That's one way to catch 'em all...\\ud83d\\ude0f","created_time":"2016-07-18T15:44:05+0000","id":"228735667216_10153763391147217"}],"paging":{"previous":"https:\\/\\/graph.facebook.com\\/v2.7\\/228735667216\\/posts?limit=10&since=1468870600&access_token=1433425760212560|n9Uq6G-sxL4iEBhUsI5VImWqpyU&__paging_token=enc_AdDIbA6qBzp1oOSdZBSAkK2tA6IRCa0E9IIjX5MJzuXFBgQstGpbwWSlGQbYpkSh6oidPhTXwehxmgcGkHVTfsjZC2x3IQ7EtlcTBHJeQKZCpHzdAZDZD&__previous=1","next":"https:\\/\\/graph.facebook.com\\/v2.7\\/228735667216\\/posts?limit=10&access_token=1433425760212560|n9Uq6G-sxL4iEBhUsI5VImWqpyU&until=1468856645&__paging_token=enc_AdCzyWMamSEswXvH9QJK7bTQhlE75j3NlsM3enODj2KLsknfgmqYuFHgfdpLZCBOuaIucBgSS6OJ34TM2j8UDZBPkldqZBLgxBIqRZCcUb98UipkDAZDZD"}}
6

I can’t reproduce your error with the JSON you posted, but I can confirm that JSONItem handles it incorrectly while both JSONItem_MTC and the new framework calls handle it properly.

7

@Kem Tekinay I’ll try the new framework but just for your information I can tell that the code fails here.

New project
1 textarea
1 listbox
1 button with this code

[code] Dim s As String = TextArea1.Text
Dim js As New JSONItem(s)
Dim results As JSONItem = js.Child(“data”)
Dim n As JSONItem

ListBox1.DeleteAllRows
For i As Integer = 0 to results.Count - 1
n = results.Child(i)
if n.HasName(“message”) Then
Listbox1.AddRow(n.value(“message”).StringValue)
End if
Next
[/code]
and the JSON data copied and pasted from this post.

Xojo 2016 version 2
OS X Version 10.11.5
System Language DK
Project Language Default

8

Off the top of my head, the rewrite would look like this (not tested):

  Dim s As Text = TextArea1.Text.ToText
  Dim js As Xojo.Core.Dictionary = Xojo.Data.ParseJSON(s)
  Dim results As Auto() = js.Value("data")
  Dim n As  Xojo.Core.Dictionary
  
ListBox1.DeleteAllRows
  For i As Integer = 0 to results.Ubound
    n = results(i)
    if n.HasKey("message") Then
      dim rowData as text = n.Value("message")
      Listbox1.AddRow(rowData)
    End if
  Next
9

@Kem Tekinay Thanks a lot - just had to move the parentheses in line 3 to get it working :slight_smile:

  Dim s As Text = TextArea1.Text.ToText
  Dim js As Xojo.Core.Dictionary = Xojo.Data.ParseJSON(s)
  Dim results() As Auto = js.Value("data")
  Dim n As  Xojo.Core.Dictionary
  
  ListBox1.DeleteAllRows
  For i As Integer = 0 to results.Ubound
    n = results(i)
    if n.HasKey("message") Then
      dim rowData as text = n.Value("message")
      Listbox1.AddRow(rowData)
    End if
  Next
10

Ug. I do that too often. Glad to hear it though.