@ChristianSchmitz A new build is here:
The cache is now much better and just stores the value actually contained in the cell. (no more extra memory reserved for possible other value types). Strings are stored in UTF-8 to make them compact. And NULL values take no additional storage.
So we are down to about 500 MB for my test project with 10 million values.
Thanks Christian. Yes, it is much better than before, though still not as lean as Xojo's SQLite class. Here are my updated results:
100,000 rows causes entire app to use 77 MB
1,000,000 rows causes entire app to use 612 MB
10,000,000 rows causes entire app to use 5.8 GB
19,400,000 rows causes entire app to use 11.2 GB
Compared to what I posted before, this is about 50% more than the Xojo SQLite class. But still better than 500%.
I didn't post this before, but the MBS class is also slower (when AutoCache is on, verus Xojo SQLite class). And this hasn't changed much with the update. Here is a comparison between Xojo SQLite and the latest SQLDatabaseMBS:
Xojo vs MBS
100,000 rows: 0.7 sec vs 3.4 sec
1,000,000 rows: 2 sec vs 13 sec
10,000,000 rows: 69 sec vs 110 sec
19,400,000 rows: 138 sec vs 215 sec
Christian, I'm still grateful for your class, as it allows me to process almost unlimited SQLite records without memory restraints when AutoCache is off. Thanks again for your dedication to the Xojo community!