If you create a new multi-resolution aware picture by passing an array of other pictures to the new picture( width, height, picts()) inside a thread the resultant raster data structures created are not released until the thread ends. If your thread is looping and creating a lot of images the way mine was you will eventually hit a state where you get out of memory exceptions.
I can create my own autorelease pools and release them every so often and the memory is recovered, but shouldnt that happen at some point in the thread anyway?
To make it build up all you need to do in a threads run event is:
dim workImages(1) as Picture
while true
workImages( 0) = new Picture( 1000, 1000, 32)
workImages( 1) = new Picture(500, 500, 32)
//
// this is the line that causes the leak
//
dim p as new Picture( 500, 500, workImages)
app.SleepCurrentThread( 250)
wend
the data backing that new multi-image picture will not be recovered until your thread ends. Is this just something about memory management in threads that I didnt know about? Or is there a bug in that new picture with an array that should have added them to a different autorelease pool on the Mac?
The code above in a thread’s Run event does not seem to leak. I’m on OS X. The memory consumed is around 20 MB and is staying at that value for several minutes now.
Sorry, yes, the memory leaked doesnt show in activity monitor. It shows in the Leaks detector of Instruments though. The example reaches a gig of allocated CG Raster Data in just about a minute. The allocation of the new picture will eventually fail with an out of memory exception even though the memory used is in the core graphics portion of system memory and doesnt seem to count against the app in the activity monitor.
@James Sentman
Please try it without the bitdepth, because the way you’re creating it, I think you’re getting pictures with masks pushed into an image that assumes that they don’t.
workImages( 0) = new Picture( 1000, 1000)
workImages( 1) = new Picture(500, 500)
I’m not saying it not a bug, but I don’t think I’ve seen any examples in our code that does it the way you describe.
I was thinking about this yesterday and concluded that it may not be a bug. autoReleased typically get released at the end of the cycle, but in this case the thread is existing beyond the cycle, autoRelease doesn’t mean now, it means whenever.
Most factory functions in objC enable autoRelease, anything with a unit doesn’t and needs explicit releasing, at least that’s my understanding.
Recently I tested an idea of using a timer instead of a thread, while may not be perfect, objects would get released.
The framework should wrap its internal code that uses Objective-C inside of an autorelease block to prevent the objects from accumulating. It’s worth a bug report.
[quote=284491:@Greg O’Lone]Please try it without the bitdepth, because the way you’re creating it, I think you’re getting pictures with masks pushed into an image that assumes that they don’t.
[/quote]
Tried it without the masks and they still do the same thing.
The whole problem would be solved if I just let the thread complete and created a new one for every cycle, perhaps that is the way they are really designed to be used. The project that bit me has been running non-retina perfectly without any memory management problems, but as soon as I started creating the image with the retina version of the data in here it would eventually start throwing out of memory errors. At the moment Im going to manage my own autoreleasepool for the thread rather than re-create the thread every few dozen milliseconds. Actually… can I just re-run the same thread object? Or do I need to create a new one… Ill have to experiment
In the app proper it just sleeps until it has some drawing to do and then wakes up and does it and goes back to sleep. In a busy system it can be working quite hard and the re-drawing needed to be removed from the main thread to keep the UI responsive. Since its a new issue I was thinking it must be a bug but really I think that the new retina objects are a direct backing to a different core graphics picture than the regular images are and so would require different memory management.
Does a thread ever release its autoReleasePool while its running? Or is that only done when the thread ends? Right now Im not seeing any other consequences of creating my own and releasing it every 5 seconds while doing the drawing. I dont even know if there is a way to specifically tell OSX to release those backing rasters when the object goes out of scope or if the only way is to clear the autorelease pool. I suppose you could not allocate them into the pool in the first place and manage that yourself like you do the higher up objects, but im not even certain if the raster memory allocation is something you have to do or if its lower down in Apples code and there is no way for you to even get to those pointers or handles directly to release them? If you cant do that then it might make sense and be necessary to provide a method on the thread to recycle the autoReleasePool. I am happy to do it through the MBS plugins at the moment But it is a gotcha for the thread that there might not be a better way to deal with.
I’m going to stick my nose in here and admit that I know virtually nothing about auto-release pools and some of these more lower-level inner workings. I’ve also not bothered to write any retina aware code yet, but I’m still interested in trying to help. So I’m not purporting to be an expert in this at all…
What if you had your code something like this instead:
while true
dim workImages(1) as Picture
workImages( 0) = new Picture( 1000, 1000, 32)
workImages( 1) = new Picture(500, 500, 32)
//
// this is the line that causes the leak
//
dim p as new Picture( 500, 500, workImages)
app.SleepCurrentThread( 250)
wend
Now workImages goes out of scope with each iteration of the loop. I would think that should clear anything out of memory. If there’s not a reason to access the array outside of the loop, then why should it exist outside the loop?
Again, maybe I’m not seeing the full picture since this is not your actual code. But why not just completely reinitialize everything each time you loop through your thread.
Also, some comments you made above about your thread makes me wonder if it might be better to use something like the ThreadPoolManager class that someone around here created (I forget who it was - Aaron Balman maybe?). In that, the thread pool manager thread just sleeps until a new thread is needed and called. These new threads exist just for a short time and you use what you need.
[quote=284797:@Jon Ogden]Now workImages goes out of scope with each iteration of the loop. I would think that should clear anything out of memory. If there’s not a reason to access the array outside of the loop, then why should it exist outside the loop?
Again, maybe I’m not seeing the full picture since this is not your actual code. But why not just completely reinitialize everything each time you loop through your thread.
[/quote]
The demo project is paired down to the least amount of code needed to show the problem and has very little to do with the original project The actual project is a database/graphing app that is connected to the home automation software that is my day job. Every time a unit in the database receives a new datapoint the graph that displays that is queued up for redrawing by this thread. This thread is woken up when the queue has something in it and does all the work of re-creating the graph. The image that it creates does survive outside the thread as its displayed by the windows or sent to the main app or the web interfaces, but only 1 image exists in memory for each graph that is being displayed at that moment. So in my graph the raster image data does stick around, but only one instance for each graph picture being displayed. It is replaced by the new one when the thread redraws it and sends the windows or any other viewers a signal that they should refresh their display with new data.
That has worked very well for non-retina images. I was working yesterday to add support for retina sized graphs and walked away to refresh my coffee while it built some graphs and had a log full of out of memory errors when I returned just 5 minutes later and thus began all the work to figure out what was going on. I hadnt changed much, just the resolution and size of the images I was creating and then creating this new multi-member image instead of the original single image. Narrowing it down I could see that the act of creating this retina image from the array of other images was what was causing it and I could clearly see it in the Instruments leaks check display.
The problem is that letting the picture object go out of scope does not return the core graphics allocated memory for that image. That is probably well known to people who write ObjC all day, but its counter to everything that Xojo does as we expect our objects to be cleaned up when they go out of scope, if you run that code above without the creation of the new retina image array image then it can run forever without any problem. But when you add that in it eventually conks out. The demo app running here has eventually either just hung up, or started throwing exceptions or even just quit with no message or error or even a crash report coming up so its very strange.
It is probably limited to OSX and its probably limited to threads, but i havent tested that in the main loop, Ill probably experiment with that shortly to make sure that doesnt cause problems there too.
So I COULD just re-create the thread every time I need to do some drawing. That is another work around that would work as the memory is returned when the thread ends. But the behavior is so counter to what I would expect from reference counting as opposed to garbage collection that it needs at least some documentation and some way to force the thread to release those things that are out of scope and given the philosophy of memory management in xojo its just wrong to have to do that. Pulling the threads from a pool of threads that are re-used probably wouldnt solve the problem. Just putting the thread to sleep and waking it up to do more work later does not clear the autoReleasePool. If I let the thread object stop instead of sleeping then it most likely would do the release. Im not sure about the overhead of re-starting the thread or re-creating the object or even if that would be a problem for the app, but it will certainly be more than just sleeping the thread. Ill experiment with that though, it might be all thats necessary. Its still wrong for xojo not to clear the memory of an object when it goes out of scope, threads or not.
A possible workaround for the time being is to side-step the multi-member image and make just the 2x Picture with it’s resolution set to 144 and scale 2. This should draw at the right size on a 1x or 2x screen, being scaled down every draw on the 1x screen. Not efficient, but if you never use the 1x version…
dim p as new Picture( 1000, 1000 ) //for a 500x500 point picture
p.HorizontalResolution = 144
p.VerticalResolution = 144
p.Graphics.ScaleX = 2
p.Graphics.ScaleY = 2
I’m not clear on all the machinations of autorelease pools and their interplay with Xojo, but maybe there’s a way to crawl into Picture.Handle and force release it. hmm I guess that’s too shaky for the solid craft you’re doing
Im experimenting with the p.graphics.ScaleX/Y settings, but that doesnt seem to affect the size that the image reports itself… but then Ive been commenting out and adding so much code at this point that I have no idea whats going on. Putting it to bed until tomorrow and Ill try to figure out whats going on. Though the kids school was just cancelled due to hurricane Hermione, or actually herman or whatever it is out there I was actually rather disappointed that its not Hermione Id be OK with that hurricane running over me
ScaleX/Y only affects drawing into the picture. Set to 2 means you can continue to draw in the 0-500 coordinate range and they get scaled up to the 0-1000 pixel range of the actual picture.
Horizontal/VerticalResolution controls the size the picture is drawn at. 72 means 1 pixel per point and 144 is 2 pixels per point.
A complication I forgot to mention is that Picture.Width/Height will continue to be 1000, that always reports pixels. But ScaleX/Y = 2 does change Picture.Graphics.Width/Height to 500. So any code that uses Picture.Width/Height to know the ‘size’ will need to be changed to use Picture.Graphics.Width/Height. Or maybe just divide Picture.Width/Height by 2 or maybe add extension methods for the point width/height
Function PointWidth( extends p As Picture ) As Double
return p.Width * 72 / p.HorizontalResolution
End Function
They don’t. Graphics.ScaleX and ScaleY affect things being drawn onto that Graphics object. If you wish to affect how the image draws, either adjust the picture’s HorizontalResolution and VerticalResolution (which are honored when the HiDPI flag is on) or when calling DrawPicture.
So I’m not purporting to be an expert in this at all…