This is totally suboptimal. How does anyone expect of us to change to the new framework? The transition will need to be way easier. Or Xojo will go the way of the dodo.
@Michel: dictionaries are hardly an advanced topic.
Fine. Then why all that fuss about the rather simple fact that keys must have the same data type, and the same encoding when they are strings ? Is it that surprising for a strongly typed language ? Heck, the same kind of feature in Swift is just the same.
I have spent the last 10 days working exclusively with xojo.core.dictionary. I love its apparent simplicity and speed. However, I have spent half of that time scratching my head about why my apparently simple code is generating errors and finding workarounds. Among other things, my working code broke when I saved the dictionaries to disk as JSON and then restored them with ParseJSON. It turns out that the dictionary you create in memory <> the dictionary you restore from disk. There is the case-sensitivity difference (annoying and totally unexpected, but at least it’s in the documentation), but another is encoding. You can happily assign strings to keys and use them when the dictionary is in memory. But once it’s written and restored, the keys are Text, breaking code that worked fine before. If you search this forum you’ll find many veteran Xojo coders (see Bob Keeney’s post earlier in this particular thread) who express confusion and frustration with the xojo.core.dictionary because it is not intuitive and difficult to debug. That’s what the fuss is about. I suggest you spend some quality time with xojo.core.dictionary in a real project. I’m committed to using it, but it certainly does not obey the principle of least astonishment.
I remained pretty silent throughout this whole thread, but please do not infer I have not spent time using Xojo.Core.Dictionary. Fact is, I use it fairly often i my apps, just as well as several other key pieces of the new framework.
Seems to me a good deal of what you describe has less to do with the object itself than with how you have been assuming it to work. As my short example above shows, classic dictionary is itself just as encoding sensitive.The example of the dictionary being different when recreated from disk data is not inherent to Xojo.Core.Dictionary, but to the way you restored it. It is not even a method of the object, is it ? If you want keys to be string, then simply assign them as such. Don’t blame the dictionary which takes what you hand to it. (ParseJSON as I understand it). It is probably smarter to use only text with the new framework anyway.
What you call intuitive often comes from habit. It is normal for the new to be unsettling and to require reading.
Saving and restoring via GenerateJSON and ParseJSON is not “my way”, it’s the intended way. Have the dictionary behave differently before it’s saved and after it’s restored is not an obvious outcome. Apparently you don’t think the principle of least astonishment matters, which is fine. I don’t see the point in responding to your inevitable reply.
“Inevitable reply”, as it is polite in any civilized conversation not to ignore others.
Fact is the new framework usually expects and generates Text. It is unfortunate that Xojo.Core.Dictionary accepts string. It always better to use only the Text type with the new framework.
Good luck anyway.
I hate to break it to yo, but JSON is not the intended way to save a dictionary. json only supports strings, booleans, numbers, nulls and json objects. It cannot represent any other data type. So the failure is not in Dictionary, rather that JSON is insufficient for representing one. And by “strings” I mean a string of characters, not the data type.
Thanks, that’s good to know (although those data types are in fact all I need to deal with, so GenerateJSON/ParseJSON seemed the simplest way). What do you suggest as a more robust method that won’t alter the key type? Is it something that we should design ourselves or is there some standard method we should be using?
you would have to make your own.
[quote=268651:@Jonathan Ashwell]This is what I see in the debugger.
[/quote]
Ah, this just the debugger showing you how the value breaks down in terms of grapheme clusters and Unicode code points.
I’m going to throw a few things out here for clarity. It’s ended up being a lot longer than I set out to write, but I think it’s all relevant here. Hopefully it won’t devolve into a flame war and now’s a good time to re-read the forum etiquette guide.
Xojo.Core.Dictionary
Xojo.Core.Dictionary is currently implemented with a red-black tree. This is slower than a hash table in the average case but allowed the class to allow custom key comparisons without also implementing hashing. This seemed like a decent tradeoff because the framework could still use a hash table internally when there was no custom CompareKeys implementation.
However, it turns out that strict weak ordering is hard to do correctly with heterogeneous data types. This is the underlying cause of a serious bug when mixing Text and String keys with a dictionary returned from ParseJSON (case 43942).
For both those reasons, it’s very likely that likely that the CompareKeys event will go away in a future release and be replaced by an event to hash a key and another event to compare two keys for equality. Correct hashing is still nontrivial and it’s likely that there will be additions to the framework to make it approachable.
Auto
Auto mimics what would happen if you explicitly specified the types at compile type. Anything disallowed by the compiler raises an exception when attempted at runtime.
Auto is extremely efficient when compared to Variant and can avoid costly heap allocations in the majority of cases, as measured against the IDE’s use of variants. Comparing the type of an Auto is currently noticeably more than Variant but will get closer in 2016r2 with optimizations to introspection metadata and the ability to use GetTypeInfo with intrinsic data types (case 43636).
Auto accurately tracks all type information for the stored values. Variant throws away a lot of type information with integer types (and possibly others that I forget).
The exceptions raised by failed Auto conversions lack any detail and make debugging more frustrating than it should be. This will be addressed in 2016r2 (case 44064).
The results of Auto’s simple rules are sometimes surprising and unintuitive. A few good examples:
// Case 39308
// The IsA operator always works with objects, so the left hand side is
//converted to an object. Auto throws exceptions when trying to convert
// non-object value to an object.
//
// This does not happen for Variant due to a fluke in the implementation.
Dim a As Auto = 20
If a IsA Dictionary then
End If// Case 41661
// - au gets created from a String and Auto remembers that fact.
// - The comparison of an Auto against a text literal has the common type of
// Text.
// - au is converted to Text in order to perform the comparison.
// - An exception is thrown because au knows it was created from a String and
// String cant implicitly convert to Text.
Dim tmp As String = "abc"
Dim au As Auto = tmp
If au = "abc" Then
End IfThe latter is actually what’s being ran into here and I consider both of these to be language defects.
Text/String Literals
The type of a quoted literal is contextual, based off of the literal’s usage. From the language’s perspective text literals:
- Implicitly convert to String with no conversion penalty.
- Implicitly convert to Text with no conversion penalty.
- Implicitly convert to an Auto containing a Text.
- Implicitly convert to a Variant containing a String (for backwards compatibility).
The type of a quoted literal can explicitly chosen using CType, an explicit type on a constant, or assigning it to a local variable and using the local variable.
Xojo.Data.ParseJSON
The decision not to have a subclass of Dictionary be returned from ParseJSON was deliberate and thought out. Such a class could always be added to the framework without breaking code, so its addition would be based off of user feedback and real-world usage. There’s been enough of that now to say that it should be added.
The Dictionary returned, regardless of the presence of an explicit class, violates the Liskov substitution principle. This is unfortunate but holds true of any Dictionary using custom comparisons.
Generating JSON and then parsing it back is fundamentally lossy because JSON’s type system is so limited. It is not a general-purpose serialization mechanism if you need to preserve type information.
Using the New Framework (non-iOS)
The classic framework isn’t going away anytime soon and it would surprise me if it’s removed in the next ten years. Once there is a new desktop framework, it’ll simply be that the classic framework stops receiving new features and bug fixes (depending on the severity).
Interoperability is an important goal because it’s absolutely unrealistic to port any sizable application over at once. This will get easier in time as the new framework is more fleshed out and additional conversion helpers are added.
Bugs
There are bugs and they frustrate us as much almost as they do you. All of us care about the quality of the product and it’s disappointing when portions of it fail to live up to expectations.
Thank you, Joe, for laying this out. It’s good to be reminded that there’s a lot going on under the surface of Xojo that we as users don’t or can’t appreciate. Letting us understand the issues and tradeoffs that lead to the current behavior goes a long way to alleviating my concerns, and I really appreciate it. As I said earlier in this thread, there is a lot to love in the new xojo.core.dictionary class, and my new code is cleaner and easier to maintain because of it. I’ve managed to work around the issues I’ve run into, there are no showstoppers for me. Since this is a particularly important class for my app I am eager to see how it evolves. Many, many thanks to you and the other engineers who are doing the hard work and yet take the time to respond, politely no less, on the forum.
Wow! Thank you, Joe, for these explanations.
If there were a weekly webpage called something like technicalinsight:xojo:com which such in-depth information about the inner workings of the Xojo framework and tools I’d be hooked!
[quote=268895:@Eli Ott]Wow! Thank you, Joe, for these explanations.
If there were a weekly webpage called something like technicalinsight.xojo.com which such in-depth information about the inner workings of the Xojo framework and tools I’d be hooked![/quote]
Unfortunately I find that writing ‘on demand’ requires a lot of effort and a specific mindset. When I have to write blog posts I tend to find myself staring at the screen and wishing I knew how to write (or that I was doing just about anything else at that moment). It’s a lot easier to hop into an interesting forum post to answer a question or just be horribly pedantic about things.