I am having great success in bringing an older app up to a 64bit target. Its running great in testing but Ive noticed that its memory usage is creeping up and up which since this is an app that runs for many months at a time without restarting, sometimes even a year, its not acceptable to be leaking large amounts of memory.
Running through the runtime object I see that keyNotFoundExceptions and outOfBoundsExceptions are not being cleaned up. In the last 3 days of running the app on my testing server Ive gained 270,000 keyNotFoundExceptions! This is new since compiling for 64 bit. Building a simple test app I can show that keyNotFoundExceptions work fine when compiled for 32 bit but never go away when built for 64bit. It seems that not all exceptions leak though, I was unable to make nilObjectExceptions leak, but I didnt test them all and I cant get outOfBoundsExceptions to leak in the test app even though they are in my real app. Only 103 of them in my app so far, that doesnt happen as often. I am actually using the trapping of keyNotFound errors as part of the logic in places so that happens much more often. I havent tested to see if 64 bit windows or linux has the same problem so Im posting this in the MacOSX section.
As an interesting aside, after first clicking on the button to generate 100 keyNotFound exceptions the runtime object also contains a reference to a single MDIWindow. I thought that was a Windows convention that I should never see on the Mac? Im certainly not purposefully creating one, why is there an MDIWindow in my app?
Yes, I also noticed 64bit compiled are using a lot more memory estate. It does not seems to release memory when possible and the apps mostly use more and more memory when using it.
Especially exceptions do have impact on memory leaks in my findings. try/catch also does leak memory so it seems.
For my apps it isn’t that ‘important’ because they mostly are running for a short time and launch again when the users needs it.
But I do think it can be problematic for background apps.
Exceptions being leaked on 64-bit is a known issue that was reported after 2015r4 was shipped, though it goes back as far as the first alpha of 2015r3.
True, but eliminates a possible optimization. For example, in one section of our code, we trap for the exception rather than use HasKey because we expect it to happen so infrequently that it would be less efficient to check each time.
Huh… I never even begun to think about using an exception as an optimization. I wonder if it really is quicker, I have to confess that for most of the time I simply use .lookup.
I have a lot to do today, but curiosity has gotten the better of me!
Okay… So I create a dictionary of 60,000 entries and then try to remove 20,000 random entries ( rnd * 60,000 ).
On 32-Bit, the difference is 20,000 ms (2012 rMBP), while on 64-bit the difference is as low as 5,000 ms.
The capturing exception error, is quicker than using .haskey - but it does depend on how many failures, if it fails every single time, then it’s exponentially slower.
There might be other costs involved too. For example, if there are threads involved that might remove a key, you’d have to wrap access in some type of semaphore, check for the key, remove it if present, then release the semaphore. In the end, you have to measure the time it takes to do all that against the time it takes to raise one exception.
Let’s say n times performing the steps to that process equals raising one exception. If you expect failure to occur 1 out of every n times, it’s a wash. Anything more frequent and you should go through the process, but anything less frequent means it’s more efficient to trap the exception.
To button this up, just comparing HasKey to trapping the exception, it’s a pretty low bar. If you expect fewer than 1 in 5 failures, trap the exception. I tested this with both the old and new Dictionaries with 10,000 entries.
It’s about the same in 64-bit, but take that with a grain of salt since, as mentioned, the exceptions are being created but not destroyed.
I am currently reworking the code not to use exceptions but what a pain in the neck. So Ive made another duplicate bug report. Why didnt anything like this come up when I searched on exceptions in Feedback?
It always makes me very nervous when even small memory leaks are treated with such casual disregard. I understand that it doesnt ultimately cause trouble for most programs since they arent run long enough. In my App in now 4 days of running Ive got almost 350,000 key not found exceptions and the performance is starting to suffer a little.
I can work around it, but a lot of effort Im sure has gone into the support of exceptions and in the documentation they describe how to properly use them. Its useful to have errors fall upwards in the calling stream until the level at which you want to handle them. Its not absolutely necessary, there are other ways of doing it of course, but this being a bug for this long effectively makes them useless when compiled for 64 bit. I cant use them at all now as part of the logic stream, only for catastrophic recovery in the global handler.
Instead of returning a value or throwing an exception I am now returning a boolean and accepting a byreferenced value holder along with the key. This works well in some places in the code, and in others Im just pre-checking the hasKey before accessing. I can get it working again, but its 2 days work time lost and thats frustrating.