I have a class containing many properties, mainly integer, Uint8, Uint16 and UInt64.
It also contain an element of another class, which itself has properties (same kind as above).
In each class, I have a string property that may be used (default “”).
All in all, each instance of the main class consumes 200 bytes (according to my count).
But when I run the app, it appears to be 4.2 to 4.5 times more.
As I may have a million (or more) of these instances, the multiplication factor is a real concern.
I build a these instances in an array and use the Append method to create a new one.
The class also have methods. I do not know if it makes a difference.
Any idea where the extra memory consumption can come from?
a class normally needs 24 byte of memory plus data size.
When counting, please include padding for C structure.
Strings take something like 24 bytes plus the bytes for the data.
Also you need one pointer for the method table.
Would it be more efficient to use Integer everywhere and concatenate my int8 and int16 in these integers?
Even if I add all the additional memory listed above, I do not end up with 4x the expected size.
I need to do processing on all of these class instances (usually one by one or at max a block of 3 consecutive ones).
After processing, display and navigation is done one by one and is therefore suitable for a SQLite implementation.
But would this save me some memory?
And what about the storage / access time?
Loading a file with 1 million entries is already lasting a bit (tens of seconds).
It should not last longer than what I have today.
One issue here is the inclusion of a string
If it were a collection of integer types, I would define a structure.
Then create a memory block of
To access the Nth element, you get back a memory block starting at position N-1 * sizeofstructure
Instead of having class methods, have module methods that act on a structure.
But those strings… variable length structs dont really work for this approach.
That’s a good idea.
I could put the string in a separate array so it will not “pollute” the structure.
But, still, I’d like to understand why I have the 4x factor on memory usage.
Understanding a problem is a good start not to reproduce it…
Is there a way to see or get the memory dump of a class instance?
So…
I got rid of the strings in my classes.
I also rationalised everything I could.
Besides moving to structures instead of classes, I do not know hot I can further optimise.
I got the following numbers:
WIN32 app -> 662 bytes per class instance (about 3.4x what I effectively use)
WIN64 app -> 958 bytes per class instance (about 4.9x what I effectively use)
I was not expecting the 64bit app to be that memory hungry.
I did not leave any of my integer to be “Integer”. They all got a defined size so the Integer becoming 64 bits instead of 32 does not play any role here.
It seems that the only way to get back control on consumed memory is to go very low level (structures or even a bunch of isolated variables).
But this does not make the code easily manageable nor readable.
[quote=248918:@Xavier Lambrecht]So…
I got rid of the strings in my classes.
I also rationalised everything I could.
Besides moving to structures instead of classes, I do not know hot I can further optimise.
I got the following numbers:
WIN32 app -> 662 bytes per class instance (about 3.4x what I effectively use)
WIN64 app -> 958 bytes per class instance (about 4.9x what I effectively use)[/quote]
How are you measuring this?
[quote=248918:@Xavier Lambrecht]I was not expecting the 64bit app to be that memory hungry.
[/quote]
Pointers double in size, so references to things like objects, strings, arrays, autos, and variants all double in size. This typically isn’t a problem because the address space is gigantic and the only real concern is paging to disk.
[quote=248689:@Xavier Lambrecht]After processing, display and navigation is done one by one and is therefore suitable for a SQLite implementation.
But would this save me some memory?
And what about the storage / access time?
Loading a file with 1 million entries is already lasting a bit (tens of seconds).[/quote]
If you’re completely in control of the fie format and can switch, SQLite is almost certainly faster.
Thanks for the feedback.
I measure the memory usage with a Runtime.MemoryUsed call. I do it before and after creating the class instances. I assume that the difference gives me the memory used by my class array. It’s maybe not 100% accurate but it seems to be quite reproducible so I take it as a valid measure.
I am in control of the file format. The only unknown is how many events (class instance) will I receive. Its not possible to derive it from file size as the file may contain different events of different size.
I will definitively have a look to SQLite.
So how much memory are we really talking about? If each class instance consumes 900 bytes (your expected 200 bytes times 4.5), you’re dealing with less than a gigabyte of memory. Is your app running out of memory? And even if the 64-bit version uses twice the memory, it has much more available to it. This feels like premature optimization. Unless you’re getting out of memory crashes, that is.