Optimize filling a Dictionary

What does the profiler say?

The profiler won’t help here as this is all happening in one method.

However, by replacing the assignment with test.Value( x ) = nil, the time is cut to about 1.3 seconds (classic) or 1.9 seconds (new) so we know most of the time is being spent in concatenation of the text.

this may sound silly at first, but maybe you’d like to give this a try:

dim keyText as text = "key" dim space as text =" " dim somevalueText as text = "Some value" For x As Integer = 1 To 1000000 test.Value(text.join(array(keytext, x.ToText), space)) = text.join(array(someValueText, x.ToText), space) Next x

(I have played around with benchmarking text and string methods and found join to be a lot faster than “+”: https://dl.dropboxusercontent.com/u/21200221/Xojo/Text%20Append%20Benchmark.zip)

That takes longer. :slight_smile: But still, it’s up to 7 seconds with the Classic Dictionary, still nothing like what Johan is seeing.

[quote=256576:@Johan van Breemen]Well, I tried it again, but in my environment (same as on which PureBasic example is tested) I get the same long time.

To be sure you and I are testing with the same example, I’ve made a test project:

Test project download[/quote]

  1. On a bit of an old dog G31 board, Win7 32bit.

To clarify, "9"… seconds, right? Not minutes?

Text is WAY slower than string. I did not compare classic and new dictionary, but overall, the new framework is generally slower.

No idea - I just ran the test project which said “9”. Seemed like seconds - definitely not minutes.

Looked at the code - seconds.

Changing to str(x) brings it to 5.

Yes, Text and core.Dictionary are a bit slower than the old String and Dictionary types. But I found the old:new framework ratio going down to less than 1:10 compared to 1:300(!) sometimes when you strictly use constants and join instead of + for the concatenations.

For long concatenations, a Join is definitely the way to go. But if there is a lot of repetition, you will be penalized for the constant creation of new arrays and the overhead of calling the Join function repeatedly.

Yes, I see that now. Just joining two small text values brings about 50% more processing time (6 against 9 seconds on my iMac). Ok, I withdraw the proposal (and still state that it’s great for building SQLite queries) :smiley:

Replacing Text with string makes not much difference.

The only big jump I just had is when using the old framework dict (11 sec to 4 sec).
Quite impressive but doesn’t makes me wonder.

The new framework isn’t optimized yet. You should file a feedback report and ask the Xojo devs
to check the xojo.core.dictionary implementation. Maybe there’s some redundant things going on in the background.

This takes “forever”:

test.Value("Key " + x.ToText) = "Some value " + x.ToText

This takes 2 seconds:

test.Value("Key") = "Some value " + x.ToText

This takes 6 seconds:

test.Value("Key " + Str(x)) = "Some value" + x.ToText

This takes 3 seconds:

test.Value("Key") = "Some value " + x.ToText + x.ToText

NB: Using the Classic Framework isn’t an option. These are new (converted) project in which I want to use the new Xojo Framework, also because iOS will be included at some point.

I already filed such a Feedback.

<https://xojo.com/issue/39570>

Johan, in your second and fourth versions, you are only creating one entry so that practically takes the Dictionary performance out of the equation.

I still don’t get why the first version takes so long for you. On a lark, have you tried restarting your machine?

[quote=256610:@Kem Tekinay]Johan, in your second and fourth versions, you are only creating one entry so that practically takes the Dictionary performance out of the equation.

I still don’t get why the first version takes so long for you. On a lark, have you tried restarting your machine?[/quote]

Yes, you’re right. I see that now. I was just testing to see if one or two ToText conversions would make a difference. Example 4 shows that that isn’t the case.

I also tried this:

txtTest = "Key " + x.ToText test.Value(txtTest) = "Some value" + x.ToText

Just to see if the problems lies in using an expression for the key parameter. It doesn’t.

[quote=256608:@Kem Tekinay]I already filed such a Feedback.

<https://xojo.com/issue/39570>[/quote]

Mmm, that’s almost a year ago …:frowning:

Sadly. I have added your test project and my results.

I still don’t get it why I have so different time results.

Maybe it has something to do with my Windows; it’s 64 bits. The test project is 32 bits. I can’t test with 64 bits project as I don’t have a desktop license yet.

NB: Making it 64 bits isn’t an option as I have to connect with 32 bits databases.

NB: Until this is resolved I’ll keep these projects in PureBasic.

[quote=256620:@Johan van Breemen]I still don’t get it why I have so different time results.

Maybe it has something to do with my Windows; it’s 64 bits. The test project is 32 bits. I can’t test with 64 bits project as I don’t have a desktop license yet.

NB: Making it 64 bits isn’t an option as I have to connect with 32 bits databases.

NB: Until this is resolved I’ll keep these projects in PureBasic.[/quote]

Checked on OSX in 32/64 bit. Difference is -1sec for 64bit.
Don’t worry. It’s the framework unfortunately. You should rather go with sqlite db instead of waiting for fixes I think.