I should process a large amount of data,
I tried with threads (from 30 to 50) but they work on a single processor and run into difficulty,
I wanted to try with the worker class but I know that the number of workers I can make work in parallel ares equal to the number of logical processors that my machine has,
so I fear a slowdown in performance
Is it possible to use workers and for each worker to instantiate 30 or 40 threads that work on different CPUs?
The workers themselves will be on separate cores, the threads within them will not.
Is it impossible for a worker to instantiate threads that make use of the CPU the worker is working on?
That’s not what I meant. Each worker instance is a physically separate xojo application and when threads are created inside a worker, they are created within itself.
That said, they don’t work that way in a debug run. In a debug run, the workers are simply Xojo Threads so you can’t actually see the performance gains until you build the app.
good , i will try with workers and a lot sons threads
do you know if there is a good example for using workers ?
Try the Xojo examples. There’s one there about processing large text files which should be close enough to your use case.
The workers are for a very narrow range of problems. If you need larger amounts of data back from the workers then they will choke. See https://tracker.xojo.com/xojoinc/xojo/-/issues/69248 . The bug gets has gotten postponed for each release in the last year.
thank you for this information
Is there a way to force the same xojo app, when instantiated multiple times, to work on a different processor than the one previously instantiated?
example
I have 4 cores
first testApp.exe >core1
seonda testApp.exe >core2
third testApp.exe >core3
The operating system is responsible for allocating CPU resources to processes - and you should let it do that, the people who develop these systems put years and years of research and testing into it. Generally speaking, they will try to maximize the amount of CPU in use at any one time so if your Workers are consuming a lot of CPU, the OS will try to give each one as much as it can take, and that will mean spreading them out amongst the available CPUs.
Note that this doesn’t guarantee that a particular Worker will be tied to a particular CPU - the OS’s scheduler will move it around as it deems necessary.
so the best approach (since the worker has problems) is an app with many threads, the operating system will move it to the most appropriate CPU
No. Workers are how Xojo lets you use multiple CPUs. Threads within a single app (no Workers) can only utilize a single CPU between them.
I did use workers before. But I never understood the “ProjectItemsToInclude” property.
Do I just enter a list of class-names, global methods, etc here?
List of Project Items to include in the WorkerConsole app. No classes or modules are included by default. Enter one project item per line. To include all classes and modules, enter
IncludeAllClassesAndModules. (Source: Worker — Xojo documentation)
Framework Classes are included by default, but not classes, modules etc. you‘ve created by yourself.
What problems?
A single Xojo app uses one core only. This is true even if the app uses threads.
A worker is a separate program. So it can use a core, and the main app can use another core. The worker can also use threads, but even so, a Xojo app plus one worker can only use two cores.
You may need to try this for yourself.
Blog on workers → Use Multiple CPU Cores with Worker – Xojo Programming Blog
As someone who’s used Xojo (and it’s previously named versions) for over 20 years, and has built a number of different applications with it. I would whole heartedly recommend looking at other development tools if you know you need this level of performance.
You don’t need to write the entire app in a different tool, you can just write a plugin which can be called from a Xojo made application.
Other development tools allow you to run pre-emptive threads, which can be used to soak every available CPU cycle on a machine, or simply run along side the main thread without freezing the GUI. Xojo’s options either require running it all on the main thread, or using a process segregation which prevents workers from accessing the main memory.
I’d also like to mention, I have one situation where a function written in another tool is so much faster than the Xojo function, I was able to move it to a efficiency core, where it not only uses less energy and memory, but still completes quicker.
So, as I asked before, Do I just add the names of those classes and modules in this “ProjectItemsToInclude” field?
Yep, only the names.
Since the workers are separate helper apps, running parallel with the main app, I can image that including all classes and modules from the main app, by using “IncludeAllClassesAndModules” would cause those helper apps to be too “heavy”. Is that right?
I mean, those apps need to start up and run. The smaller the helper app, the faster it loads, right?
For now I included all the main app’s classes and modules, for my convenience. But then the workers would loose their efficiency, I think.
It would be nice to make something that looks for what modules are being used by the workers. One way would be to figure out the modules through introspection. Christian (MBS) uses introspection for Memory Leak detection. I bet this trick can see what modules and classes are being referenced to, in a worker.