Comments on new JSON handling

General
1

I had just commented on Xojo’s recent blog post, but I thought I repeat it here again in case others want to comment:

1. Comment:

Hold on - Does the new Xojo.Core.Dictionary type guarantee the order of items added to it now? The classic one did not, and I’d be surprised if these semantics have changed.

I like to point out that, when working with JSON data, it’s sometimes critical that the order of items is known and stable. So, if I add two key-value pairs, I expect them to be kept in that order, both when convertin them to text and back. The old JSONItem could do that, apparently. I doubt that the new Xojo.Core.Dictionary does that, or will it?

2. Comment:

Yep, just as I feared:

The input text

{ “z”: “z value”, “a”: “a value” }

when converted with this code:

Dim d As Dictionary = Xojo.Data.ParseJSON(TextArea1.Text.ToText)
TextArea2.Text = Xojo.Data.GenerateJSON(d)

comes out with the wrong order:

{“a”:“a value”,“z”:“z value”}

Now, while you may claim that this is within the spec, it’s still unexpected as it shifts the behavior we’ve come to expect from the old JSONItem class.

Furthermore, the WP article says for this type of JSON data:

“Object — an unordered collection of name/value pairs where the names (also called keys) are strings. Since objects are intended to represent associative arrays,[10] it is recommended, though not required,[11] that each key is unique within an object.”

Now, what do you think will happen if I have two key-value pairs with identical keys in there, which according to above quote is valid? Yes, boom.

Though, at least in this case, JSONItem didn’t do better either.

Also, shouldn’t the new Dict classes at least provide case-sensitive keys now? Yet, when I have two keys in a JSON “object” that only differ in case, such as “a” and “A”, I still can’t convert them into a valid JSON object or dictionary, or can I now? Because, JSON keys are indeed handled case-sensitively by most JSON libs I’ve used. Only Xojo’s fails for me.

Remember: Just because it works for you, it doesn’t make it a well working API. :wink:

2

JSON should NOT depend on the ordering. It should be key value pairs - order should not matter.
It you use services where it does then I’d suggest reporting that oddity to the service provider.
I’ve run into a couple and reported that to the provider and resorted to having to create things differently (usually with text concatenation) to guarantee order.

That would be an enormous behavior change
Xojo is a case insensitive language in general
So to have one component be case sensitive would be very odd
Having a way to make it behave in a case sensitive fashion, perhaps through a user implemented comparator, might be useful though
But the default would be to be case insensitive to be consistent with the rest of the language

I’m not sure there’s no a feature request for such a thing already

3

Well, the least reason for keeping the JSON items in order is for debugging and reproducibility reasons. I am the master of my code and my environment. And if I use only code that keeps the order the way I inserted the items, then I am allowed to rely on them, ain’t I? And with JSONItem, I was also able to have that control. All I am pointing out here is that you’ve introduced a regression here. Defending that by telling me that I don’t need that and that my algorithm is badly designed is just not your position to be in.

4

As a note- you can make the new Dictionary case sensitive (or change its key handling in other ways) if you like:

http://developer.xojo.com/dictionary$CompareKeys

5

The ECMA spec and JSON.org don’t say much about writers and having to preserve order.
JSON ARRAYs are documented to be ordered collections.
OBJECT it simply says they are An object is an unordered set of name/value pairs. An object begins with { (left brace) and ends with } (right brace). Each name is followed by : (colon) and the name/value pairs are separated by , (comma).

So I would not rely on order in any case unless you’re using an array.
Even if its my own code unless you’re manually writing the entire JSON structure as strings in a specific order.

6

[quote=180752:@Travis Hill]As a note- you can make the new Dictionary case sensitive (or change its key handling in other ways) if you like:

http://developer.xojo.com/dictionary$CompareKeys[/quote]

But that won’t work with reading a JSON text, does it? I believe it doesn’t because the conversion is done by a shared function now that returns a non-case-sensitive dictionary, so I can’t tell it to use my modified case-sensitive class, or can I?

7

Norman, the text I quoted from WP points out a passage that makes it clear that JSON’s RFC allows the items to be ordered.

It would also help if you would at least admit that the behavior has changed (from old to new Xojo API) in this regard, to the benefit of other users who might get bitten by this.It’s not all about being right or wrong, but about what you do to your paying users, and how you treat them and their needs.

8

If I recall correctly, ParseJSON does return case-sensitive dictionaries. Are you finding that this isn’t the case?

9

It does, and this isn’t an issue in context of JSON. I was just pointing out that the new Dictionary is customizable in that way in general.

10

We’ve never guaranteed the order of JSON, and the spec does not either. One of the reasons the spec does not guarantee order is so it is easily movable into and out of Dictionaries/HashMaps/etc- which is how we’ve chosen to do it in the new framework. We believe being Dictionary based is easier to use for the vast majority of users. If your case requires particular order internally, then you can stick with classic JSON implementation. Most use cases do not.

11

I just tried this code with the expected, and correct, results:

  dim d as new xojo.core.Dictionary
  
  d = xojo.data.ParseJSON( "{""a"":1, ""A"":2}" )
  dim v1 as integer = d.Value( "a" )
  dim v2 as integer = d.Value( "A" )
12

Interesting: The xojo.core.Dictionary is not case-sensitive by default, but the one returned by ParseJSON is even though it’s not a subclass according to the debugger. How can that be?

13

AddHandler.

14

Kem, if you are right, and I trust you are, then I made a mistake during my tests:

I had edited the test input using a TextArea’s Text property. Due to the other problem I mentioned (editing would convert all quotes into smart quotes), I may have caused unparsable JSON data. When my test run then gave a runtime exception, I incorrectly assumed the exception was due to a dictionary name clash.

So, my apologies for my faulty test methods and wrongful claim. I am, in fact, glad to see that I was wrong on this and that JSON data is now handled case-sensitive.

15

Wow, the uses of AddHandler are scary but impressive :slight_smile:

16

So how can I tell if I am using a case-sensitive Dictionary or not other than just knowing? Wouldn’t a subclass be the “more correct” way to do it?

17

You can’t tell, which generally shouldn’t be a problem.

Possibly and it’s something that was considered when creating ParseJSON. The API contract would allow us to do this in the future without breaking any code if we felt the need.

18

BTW, I’ve not seen my two comments I made an hour ago show up, yet - I would’ve liked to correct my comment on the case-sensitivity, if possible. Or just don’t post them at all, and I just post a new one that only points out the small change in the ordering behavior. But I like to know why my posts haven’t appeared. Are you screening them and decided not to post them or does it just take longer for them to appear?

19

I removed your 2 pending comments from the blog moderation queue- feel free to post an updated one. They’re generally moderated once a business day or so.

20

I feel rather strongly about this so I created a feedback request: <https://xojo.com/issue/39002>.