I need some non-blocking file operations in my app (“Find Any File”), so I played around a little with GCD.
Turns out that with a few declares I can nicely run pre-emptive thread operations via GCD.
Of course, there are a few things to consider:
Stack overflow checking needs to turned off in the preemptive code (#pragma StackOverflowChecking false).
Background tasks need to be disabled in the preemptive code (#pragma DisableBackgroundTasks).
All preemptive code must run in global (static) methods, and any accessed variables must be global (static) as well.
Creating / releasing of Xojo objects must be prevented.
In my quick test, I ensured the last requirement by only counting up a global integer value.
But I imagine I could also use OSX based locks (e.g. POSIX mutex or maybe even something like NSLock) to make it safe to mess with Xojo objects, or can’t I? And then I perform the actual asynchronous operations (such as file operations) using declares.
I’d be intrigued to see how you went about this. I have tried a few times (not with GCD but with async callbacks on various unmanaged threads) but I always get a StackOverflow when the method is called, even before the code in it runs (including the pragma). If a way to make these callbacks work were possible similar to how you managed GCD it would open up a new world of possibilities for iOS and integration with AVFoundation/Kit and other important, but currently inaccessible, functionality.
Oh, the stack overflows I could easily avoid using the pragmas using the rules I outlined above. I don’t see how you could have trouble there.
What I’m still trying to understand is whether I can use mutexes to control access to Xojo objects from within a preemptive thread. But I’m slowly realizing that that can’t work because even if I protect access to a particular Xojo object, the thread would mess with Xojo’s overall object pool, and that can cause any kind of random damage.
So, it seems that I will have to modify Cocoa or CoreFoundation objects such as NSMutableArray to pass data around.
I have never used GCD before, and I get the impression that GCD is even overkill here, and that I might as well use the classic pthreads instead. The only advantage of using GCD for my purpose is that it saves me from handling a queue for repeatedly calling my preemptive i/o function.
I don’t ask for “supported”, I ask for reasons why this can’t work, and you haven’t provided any information on that, yet
So far, I found it to work stable as long as I avoid using Xojo objects inside the preemptive code.
The biggest pain is that one cannot use Strings (this includes String literals) and must declare every CF function by hand as one cannot use the “macoslib” classes either. But apart from that, it works, so far.
It’s a pain to use, though. I am still considering writing the pre-emptive code in Xcode and then export the functions and access them via declares. That comes effectively to the same as what I do now in Xojo but saves me from the tedious need to declare all those needed CF functions.
This brings back memories of a threading toolkit / plugin I offered briefly. At the time it seemed stable enough if you obeyed all the rules. With later versions of Real Studio it became less stable and I pulled it. That was a while back as the initial testing was on a G4 iMac
I once dusted off that project and played around with it in RS2012r2.1. It appeared that I could still get RBScripts to run in multiple threads across multiple CPUs (i.e. one script per thread, not one script over multiple threads) so long as I obeyed certain rules such as no IO events or calls to the Context object. Not sure how safe this really is though so I never shared it.
Ok I was just looking back through my most recent attempt and realized that I hadn’t included the DisableBackgroundTasks. Adding that seems to make it work. Now to see how (if) this works with some AV stuff I’ve been wanting to use in iOS
By pressing the Pushbutton, it queues up 100 async executions of the method “myWorker”, in which a global variable is incremented by one and also adds the incremented value to an array (named queue here) as a CFNumber. The Pushbbutton code then retrieves the 100 CFNumber values from the queue and shows the last one. There’s also a Timer which shows the current value of the global variable.
With each push of the button, the counter is therefore raised by 100, which is then printed into both labels. I challenge anyone to crash this.
There may be flaws in my code in the way that it’s not aggressive enough to prove that it’s actually running concurrently in both the main and the preemptive “worker” thread. I may have to look into this more tomorrow. Now I need some sleep first. Good night and good luck finding a fault in it.
Joe & I have been conversing about this
And the basics are that you have to avoid so much of Xojo as it MAY call into the runtime, a framework or a plugin that is not preemptive thread safe that you might as well do it in C
This is the beauty of preemptive threads.
You cant “test them” into being correct.
And since so much of the framework in Xojo is hidden from your view you can’t reason about it’s correctness for this user either.
A bunch of small helper apps or several instances of the same helper app are easier to debug & reason about.
And even if one crashes it doesn’t have to take everything with it etc.
On Windows, the only platform where I’m using Xojo, I never had a bulletproof working “async” call to Xojo code.
My knowledge suggested me (and proved to be true) that any async or concurrent processing must be done in an external dll or plugin that must report result to Xojo using IPC (i.e. sockets) if you want to avoid polling the async result and/or status.
This is the only reliable and stable way, for my knowledge, to get a useful result.
Thomas, way not ask me to add something to the plugin to do it right without using Xojo for the worked method?
Our plugin can already copy files in background.
I have looked at the assembler code of the functions I wrote, such as myWorker and the helper functions (LockEngage etc.). The code is clearly free of any Xojo runtime code calls (other than the exception handling which I don’t let be invoked by not causing any Xojo exceptions).
As long as there are no calls into Xojo’s runtime to be seen, there are none, and therefore there is no “risk” of something unseen happening. If I can see all the code that gets invoked from within my GCD dispatch function, then I see that it’s clean. Unless I’ve suddenly turned blind and can’t read disassembled code any more, of course.
I’ve now also made the code much more aggressively concurrent, running multiple threads at the same time, all creating CoreFoundation objects that I then read again from within Xojo’s main thread and process with normal Xode runtime functions. No crashes, for hours.
So, be the skeptics you like to be, responding only with vague statements. I claim it’s possible and the demo I’ve made demonstrates it. The code is nothing different from code I’d end up with in a plugin, where using such threads is safe, after all, if done with the same care I employed here.
Sure, I may be still wrong in this case and be overlooking something, but I am willing to discuss it and demonstrate it, i.e. being totally transparent about it as shown above. OTOH, my plea to get counterarguments are not offered by either Xojo employee. Both claim they know why it can’t work. But they do not want to disclose what they know. If they want to really help, they could just explain.
So, here am I, offering all the arguments and facts and get treated like a 3-year old that’s told about the bogeyman.
They could, but they don’t have to. And they don’t have to explain why they don’t want to. Take it or leave it.
I assume they will not give any detailed explanation, because it will start a spiral never ending and in the end you’ll be all talking in assembly to each other. Choose the right tool. Go with Objective C or Swift for such a task. Your project is not what Xojo is good at and being developed for.
The nightmare with ThreadMBS was that people used it for problematic things.
Referencing any xojo object will cause the lock/unlock calls to be added to the code.
You can avoid them with a pragma, but that makes things more unsafe.
The code with nil object, bounds or stack checking may cause exceptions which will also invoke the runtime and cause problems.
On the end, if you do only some math and invoke system objects which thread safe and leave out the runtime functions, it can work. You’ll have to not use debugger, text, string or memory blocks.
But if xojo compiler changes someday and adds another check call there, it’ll break. Or if you put some line which does more than you expect.
ThreadMBS was a nightmare to support as people tried a lot of things there which worked most of the time, but not always.
