Failure creating JSONItem

  1. 6 months ago

    Dale A

    Jan 18 San Diego, California, USA
    Edited 6 months ago

    I've been trying to figure out why my program would crash after retrieving a large amount of data from a website. I finally tracked it down to creating a JSONItem from it. I've filed a bug report and supplied a test app.

    Feedback Case #54695

    I'm currently using 2018r4 on Windows 10.

    (For what it's worth, the data received is a bit over 1 million characters.)

  2. Kem T

    Jan 18 Pre-Release Testers, Xojo Pro, XDC Speakers Connecticut

    Have you tried my JSONItem_MTC, a drop-in replacement for JSONItem? It solves some problems that still exist in JSONItem and might do the trick for you in the short term. (It helps that it's also faster than JSONItem.)

    https://github.com/ktekinay/JSONItem_MTC

    Note the project also includes functions to convert JSON to/from Dictionary/Variant array.

  3. Dale A

    Jan 18 San Diego, California, USA

    @Kem T Have you tried my JSONItem_MTC, a drop-in replacement for JSONItem? It solves some problems that still exist in JSONItem and might do the trick for you in the short term. (It helps that it's also faster than JSONItem.)

    https://github.com/ktekinay/JSONItem_MTC

    Note the project also includes functions to convert JSON to/from Dictionary/Variant array.

    Actually, Kem, that's on my agenda for this afternoon. I have it and have used it in a different project. The only "complaint" I have with it is that in the debugger, the dictionary keys show as hex strings rather than words. Not serious but sometimes annoying.

  4. Kem T

    Jan 18 Pre-Release Testers, Xojo Pro, XDC Speakers Connecticut

    Necessary since JSON keys are case-sensitive, so that's the only good way to do that. But I thought I had added some debugging property to list the keys? (I might be thinking of something else.)

  5. Dale A

    Jan 18 San Diego, California, USA

    Hi again, Kem. I just dropped the module into the test project and it seems to work properly in creating the JSONItem_MTC from the data.

  6. @Kem T — I respectfully disagree with you. I think a better way to implement case-sensitive keys would be to generate your own hash table and override some functions so the user can have the right (unchanged) keys and the right behavior at the same time. Though it may be nitpicking :-)

  7. Kem T

    Jan 19 Pre-Release Testers, Xojo Pro, XDC Speakers Connecticut

    Interesting. Potential performance issues aside, how would that work if the JSON object has, for example, the keys "my key" and "MY KEY"?

  8. #Kem Tekinay — With the Xojo's standard Variant.Hash function, both would have the same hash. But if you use your own case-sensitive hash function, the 2 would have different hashes. Dictionaries don't compare keys but their hash, for speed.

    If you are interested, you should have a look at Jenkins hash function or, more generally, to non-cryptographic hash functions

  9. Kem T

    Jan 19 Pre-Release Testers, Xojo Pro, XDC Speakers Connecticut

    I understand how hashes work. :) My question is, what would you end up storing as a value in the Dictionary to alleviate the problem of not being able to see the original keys? How would you link the key to its hash or vice-versa?

  10. Kem T

    Jan 19 Pre-Release Testers, Xojo Pro, XDC Speakers Connecticut

    I just looked at the project, and I misremembered somewhat. I did add a DebugKeysMap property to JSONDictionary, but that is only used by M_JSON.ParseJSON_MTC, and has nothing to do with JSONItem_MTC.

    I suppose I could change the key encoding so it tacks the hex encoding onto the original key, something like "my key-ABCDEF" (not a real encoding), but I'd have to check the performance hit of that.

    I could also do something like this: "mykey" stays "mykey" while "myKey" becomes "my Key" and "my key" turns into "my key" (one space turns into two) and "my Key" becomes "my Key" (each space turns into two, each capital letter gets a space before it). Something like "MYKEY" would become " M Y K E Y".

    Again, I'd have to see what the performance hit of something like that might be.

  11. Kem T

    Jan 20 Pre-Release Testers, Xojo Pro, XDC Speakers Connecticut

    I ran some some tests and if I convert the name to a Dictionary key like this:

    key = name + "-" + EncodeHex( name )

    I don't see an impact on performance and, if you examine the Dictionary in the debugger, you will see keys like "A-41", "a-61", "myKey-6D794B6579". This will keep it unique for case and make it easier to debug, yes?

  12. Dale A

    Jan 20 San Diego, California, USA

    @Kem T I ran some some tests and if I convert the name to a Dictionary key like this:

    key = name + "-" + EncodeHex( name )

    I don't see an impact on performance and, if you examine the Dictionary in the debugger, you will see keys like "A-41", "a-61", "myKey-6D794B6579". This will keep it unique for case and make it easier to debug, yes?

    What effect will that have on methods like HasKey()? If I currently have a Dictionary that has a key '696E666F' ('info') I can have a statement like

    If myDictionary.HasKey("info") then
    ...

    Will my statement as written fail? Will I have to revise it? Or will the new key format be only for the debugger?

  13. Kem T

    Jan 20 Pre-Release Testers, Xojo Pro, XDC Speakers Connecticut

    You won't see a difference until to try to manually examine the object Dictionary in the debugger.

  14. Dale A

    Jan 20 San Diego, California, USA

    @Kem T You won't see a difference until to try to manually examine the object Dictionary in the debugger.

    Sounds good to me!

  15. @Kem T — Sounds good

  16. Kem T

    Jan 23 Pre-Release Testers, Xojo Pro, XDC Speakers Connecticut

    Updated, see https://forum.xojo.com/conversation/post/422355 and continue any conversation about JSONItem_MTC in that thread please.

  17. 5 months ago

    Tim S

    Feb 14 Pre-Release Testers, Xojo Pro Phoenix Arizona USA

    This is really fantastic Kem! I had the same problem and you solved it!
    Thank you for sharing your hard work!

    Tim

or Sign Up to reply!