Dictionary Corruption

[Lee_Badham]

We are getting a very weird error when using a dictionary. Sometimes, the values become corrupted.
Here is an except from debugging the problem.

dim overalls(2) as string

dim d as new dictionary
dim pd as new mydictionary
dim d2 as new dictionary
dim d3 as new xojo.core.dictionary
me.gradedata = new mydictionary

dim overall as string = overalls(0)
dim cmyk as string = overalls(1)
dim spot as string = overalls(2)
me.gradedata.value("overall") = overall
d.value("overall") = overall
pd.value("overall") = overall
d2.Value("overall")="3"
d3.Value("overall")="3"

me.gradedata is a mydictionary class and is a subclass of dictionary. The others are just local variables for testing to compare.

overalls(2) is a string array containing (“3”,“”,“3”) and can be confirmed by looking in the IDE.

But, when I look at the dictionary, the values for the keys show as empty and the typeinfo of the value contains gibberish ‘UH��AVSI��H���!fH��L���fH�T�H�8’

We normally write the dictionary as a string to save it and have a loop to generate a string data for many of the types

dim attributestrings() as string
dim thiskey as variant
dim thisvalue as variant
for each thisentry as DictionaryEntry in d
  thiskey=thisentry.Key
  thisvalue=thisentry.Value
  
  Var ti As Introspection.Typeinfo = Introspection.GetType(thisvalue)
  
  if ti<>nil then
select case ti.name
case "string"
....
....
case "Folderitem"
  valuestring = encodehex(folderitem(thisvalue).SaveInfo(nil))
else
  if ti.IsPrimitive then
    //basic type
    attributestrings.Append(thiskey+chr(9)+EncodingToURLMBS(thisvalue))
  else
    break
    app.debugmessage("dictionary_string error mydictionary string for a type "+ti.fullname)
    valuestring="BADDATA"
  end if
end select

We get a breakpoint with it.fullname containing the above gibberish type when it goes wrong. Processing the same data again does not cause each time.

Back to the code above…
d, pd, d2 and d3 are all local variables.
Inspecting them in the IDE shows that dictionary d and d2 are both corrupt. The value of “overall” is empty and has the gibberish ti.fullname. How this can happen to a local variable, I really can’t understand. d2 has “3” directly allocated to it. d3, of type Xojo.Core.Dictionary is ok and contains a single entry with a key of “overall” and a value of “3”. Unfortunately, I can’t make mydictionary a subclass of Xojo.Core.Dictionary because there are many internal functions and plugins which require a ‘Dictionary class’, not ‘Xojo.Core.Dictionary’

Has anyone seen anything like this before?

I’m using Xojo2024r1.1 and Xojo2024r1. MBS 2023r4 and now 2024pr2 beta as a test

Regards,

Lee

[Kem_Tekinay]

I’ve not seen this and can’t reproduce your results. Here is how I tested your code:

dim overalls() as string = array("3", "", "3")

dim d as new dictionary
'dim pd as new mydictionary
dim d2 as new dictionary
dim d3 as new xojo.core.dictionary
'me.gradedata = new mydictionary

dim overall as string = overalls(0)
dim cmyk as string = overalls(1)
dim spot as string = overalls(2)
'me.gradedata.value("overall") = overall
d.value("overall") = overall
'pd.value("overall") = overall
d2.Value("overall")="3"
d3.Value("overall")="3"

This is on a Mac (ARM).

The entries are fine in the debugger, so it sounds like something else is going on.

[Rick_Araujo]

@Lee_Badham

We are getting a very weird error when using a dictionary

Create a sample, a complete one, showing the issue. Share it.

[Lee_Badham]

There’s no consistency in it failing for me. It always had worked on the first run. When the code calculates from the same job data for the second or third time it may go wrong. The local variable of type dictionary is corrupt, the local variable in Xojo.Core.Dictionary is ok. When it goes wrong, It’s always the same error with the corrupted variable type.

I’ve changed just that me.gradedata class to be Xojo.Core.Dictionary and so far it’s all working even after multiple runs. There’s still loads of other dictionaries in use elsewhere. I’ve had to add a couple of Extensions to Xojo.Core.Dictionary for the loading/saving of dictionary values.

Regards,

Lee

[Lee_Badham]

I’m testing on a Mac M2 with an intel desktop build. Due to other plugins, I can’t create a universal binary yet.

[Beatrix_Willius]

Why do you subclass the dictionary and not write extends or use a wrapper? I use dictionaries all over my app and have never seen such a corruption.

[Lee_Badham]

I did have extensions, but I was adding so many I decided to subclass it. In the example it made no difference whether I used mydictionary or dictionary. It was only with Xojo.Core.Dictionary did I not get any corruption.

[Kem_Tekinay]

Is the corruption consistent? IOW, is it the same gibberish each time? If not, it sounds like something is releasing memory it shouldn’t be releasing.

[Lee_Badham]

Yes, the corruption is consistent. Typeinfo ti.fullname is always the same thing, as far as I can tell. I’ve not looked at the binary, but the displayed string is the same.

I keep coming back to the local variables being wrong too, not just the one belonging to the overall class.

I can step line by line and see that the strings are ok, but directly after the d.value(“overall”)=“3” line the dictionary is bad when stepping through the code. Using ‘Step In’ shows that there are no context switching and each line in the debug is run.

Regards,

Lee

[Lee_Badham]

This part is being run in the Main thread. There is a separate thread with access to the class (and the gradedata dictionary), but the thread is only a reader and at the time of corruption, it’s asleep. When inspected from the thread, the dictionary is similarly corrupt.

Regards,

Lee

[Christian_Schmitz]

Can you make a project to show it and maybe report as bug?

[Rick_Araujo]

@Rick_Araujo

Create a sample, a complete one, showing the issue. Share it.

@Lee_Badham

There’s no consistency in it failing for me.

I’ve read it as “I can’t”

[Lee_Badham]

Pretty much. I’m not sure how I can re-create the problem in a small project. The post here was really if anyone else had come across a similar problem. I’ve spent almost two weeks trying to diagnose what is going wrong and re-factoring code. Now I’ve got it working (with Xojo.Core.Dictionary) I’ll need to move onto something else, but I’ll see what I can do.

[Mike_D]

In your Dictionary subclass, are you using a Constructor? Does your code call

Super.Constructor

Failing to do that can introduce subtle bugs…

[Lee_Badham]

Hmm. interesting.

I have 2 constructors, one had super.constructor in it, the other one did not.

I’ll add it and see.

It still doesn’t explain why the local dictionary(both dictionary and subclass) are wrong.

Regards,

Lee

[Lee_Badham]

Although Xojo.Core.Dictionary was working, I would need to go through all the code and replace any dictionary.lookup code as when trying to compare with a string I’m getting a type mismatch for text and string.

I added the missing Super.Constructor to one of the constructors but it made no difference.

Regards,

Lee

[DerkJ]

Did you check the encodings of the keys if they are string ?

[Mike_D]

@Lee_Badham

I added the missing Super.Constructor to one of the constructors but it made no difference.

Too bad, that would have been an easy explanation and fix.

Do you see the corruption only in the debugger or does it happen in a built app too? For example, what if you debug the info using system.debuglog() only? I could imagine this is a debugger bug only.

[Lee_Badham]

The keys look ok, it’s the values which are bad

[Lee_Badham]

In the built app, we just get an immediate crash. The App closes with no message or crash log.