Dictionary to string and back

I frequently find the need to export a dictionary with heterogeneous data to a string, so i can write to a file, put on the clipboard or send via a socket. So it would look like this:

dStr = d.toStr()

then

d.fromStr(dStr) to re-create the dict on the other end.

This seems like such a common and useful function that I’m surprised there is nothing built-in. I searched MBS and could not find anything either (but I often miss stuff when searching there).

Years ago I modified Kevin Ballard’s excellent XMLDictionary module to support all the usual xojo data types, including folderItems, arrays of numbers and strings, and child dicts, and it works well, but I was wondering if there is a modern built-in way to accomplish this that I’m missing.

You should look at GenerateJSON and ParseJSON which may help.

1 Like

Thx Wayne. I looked at this in the past and it doesn’t really fit the bill. For instance, you cannot store/retrieve memoryblocks or folderItems, and even this does not work:

Var d as new Dictionary
Var arr(100) as double
d.Value("arr") = arr
Var json As String = GenerateJSON(d, True) // With expanded format
Var d2 as Dictionary = ParseJSON(json)
Var arr2() as double = d2.Value("arr")   ' typeMismatchException

An out-of-the-box function would be very useful in my opinion, as dictionaries are a very powerful construct that I use a lot.

Playing around with your code, I’m seeing something I don’t understand.

When this line runs:
Var d2 as Dictionary = ParseJSON(json)

I would expect d2 to be a Dictionary which holds one item named “arr” whose value is an Array(100) of Doubles.

But the debugger shows that it’s an Array(100) of Objects

Is this a bug or am I misunderstanding how ParseJSON() is supposed to work?

Edit to add: from the documentation it seems like we should be getting an Array() of Variants?

See ParseJSON — Xojo documentation

JSON data types map to corresponding Xojo types as follows:

JSON Data Type Xojo Data Type
Number Variant containing a numeric type such as Integer or Double.
String Variant containing a String.
Boolean Variant containing a Boolean.
Array, noted by Variant containing array: Variant()

Something very weird is going on here.

Try running this code in the IDE:

Var d as new Dictionary
Var arr(100) as double
d.Value("arr") = arr
Var json As String = GenerateJSON(d, True) // With expanded format
Var d2 as Dictionary = ParseJSON(json)

dim v as variant = d2.value("arr")
dim objarr() as Object = v
dim foobar as Object = objarr(0)
break

When I do, the ‘foobar’ variable, which should be holding whatever is in ObjArr(0), is not showing up in the debugger at all !

image

I’ve reported this as https://tracker.xojo.com/xojoinc/xojo/-/issues/73660

JSON arrays can be of mixed types so parsing an array will always yield an array of Variant.

var arr2() as variant = d2.Value("arr")

Good insight @Kem_Tekinay

If I change the code like this:

Var d as new Dictionary
Var arr(100) as double
d.Value("arr") = arr
Var json As String = GenerateJSON(d, True) // With expanded format
Var d2 as Dictionary = ParseJSON(json)
dim v as variant = d2.value("arr")
var v2() as variant = d2.Value("arr")

I would expect that V be a variant which was an array of Variants(), whereas V2 would be an array of Variants.

Instead, here’s what we get:

image

And if you look into what the V’s objects contain, they are empty:

They aren’t empty, the debugger is just wrong.

ParseJSON and GenerateJSON are the closest you’ll get. Serialization is a hard concept, and Xojo just avoids it entirely. When developing WE, we really needed a universal serialization routine, but there was no reasonable way to make it work. I even had declares that could generate objects from strings containing the class name.

I have request https://tracker.xojo.com/xojoinc/xojo/-/issues/62292 open that has not gained any traction. It could be implemented by default on objects such as MemoryBlock and FolderItem, but this is only a partial solution. Serializing is the easy part. Unserializing is the hard part, and there’s no way to avoid having to manually rebuild the objects.

Thanks Thom - can you clarify what you mean?:

  • Do you mean it’s not actually an Object(100) array, it’s a Variant(100) array but the debugger is showing it incorreclty?
  • Or that it is an Object(100) array, but the Objects aren’t Empty, and the debugger is not showing their values properly?
  • Also, should we be seeing an Object(100) array at all? I don’t think that’s consistent with the documentation.

Thom, what is serialization exactly? (sorry if this is an obvious thing to others)

So the above was my main point: in my mod to XMLDictionary I rebuild arrays, folderitems, mb’s etc manually. But I was hoping there is a robust modern solution that does this for us. Seems not, surprising, since dict to str and back is such a useful function.

I’m not sure precisely what it is behind the scenes, but it is a 101 member array. Surprisingly, Object() is assignable to Variant(). I’ve got a theory in my head as to why this is, but I’m not going to go into it because I’m not confident of it.

1 Like

Added my :+1:

Serialization is turning objects into strings. A dictionary is an object. Unserialization is the reverse.

@Peter_Stys This way of working with JSON as Dictionary objects is kind of a new addition to Xojo, and as we are seeing, it seems to have some bugs.

The older way of doing it was to use the JSONItem class. In my experience that is more reliable and predictable than using JSON as Dictionary.

However, the JSONItem class does not do what you want: it does not have very many ways in which it will auto-convert betwween Xojo classes.

It’s not hard to write the helper functions, however, I’ve done some of those myself.

Got it, a formal name for the original concept I was describing. thx.

Here’s an example helper function I wrote, which takes a JSONItem containing an array of Strings and converts them to a Xojo String Array:

Public Function ToStringArray(j as JSONItem) As string()
  #Pragma DisableBackgroundTasks
  dim sa() as string
  if j <> nil and j.IsArray then
    dim u as integer = j.Count-1
    for i as integer = 0 to u
      dim vv as variant = j.ValueAt(i)
      sa.Append vv.StringValue
    next
  end if
  return sa
End Function

I bet you could modifiy the existing XML functions you wrote to work with JSONItem fairly easily?

Also, if you are going to and from strings using JSONItem, the functions are:

dim j as new JSONItem(json as string)
dim json as String = j.ToString

Thx Mike, I’m sure I could, but since I already have a working solution (albeit a kludgey one) I don’t see the point. The only downside of my current approach is that the module has to be included in every project and the code will likely break at some point, so i was hoping to future-proof it with something built-in. Redoing in json would just bring me full circle.

I understand and agree - I have a big project where some parts use XML and some use JSON, based mostly on my whim and the date when I wrote the code. I find JSON a lot more human-readable and easier to debug, whereas XML tends to be harder to read and more verbose, but does have more built-in features (such as CDATA ). If your XML system works, I’d stick with it.