I’ve been playing around with Workers and doing a test of the new Worker class. On a 2015 MacBook Pro that’s been Bootcamped the same file set takes considerably less time in Windows than on Mac. I don’t have the exact numbers in front of me at the moment but it was considerable.
Is the updated FolderItem for APFS that much slower? Is this an Apple issue or a Xojo implementation issue?
Anyone else seeing this?
There are definitely parts of it that are far slower. For me the biggest problem was the Swap Folderitem function in order to do a safe save. Write to a temporary file and once verified swap the temporary file with the real one. That was an order of magnitude slower on APFS.
Also dealing with folderitem objects is very different. They no longer still point to the same file if you rename any other folder in the path. They really are just a dumb path now. This caused me no end of frustration to figure out why things were sometimes broken after the update.
I haven’t noticed that otherwise just handling files without those things are any slower, or enough slower to actually notice. There are almost certainly other implementation differences we can blame between windows and the Mac. Do you have the same problems when running not as a worker? Is it worse in debug mode where the worker is running as a thread and not as a separate process?
I was only dealing with Workers. I should redo the rest in the main app and thread for comparison sakes.
At work we deal with a lot of files. If newer FolderItems are considerably slower that’s going to be a problem.
I once wrote functions like ReadFileMBS and WriteFileMBS for my plugins to speed up a few of my apps. For reading thousands of text files and writing thousands of small html files, it’s much faster.
Not to mention ReadFileMBS() is just so dang convenient for one-line reading of a text file into a string variable. Sure, I could write functions for some things in MBS. But why??
So I’ve done some more testing on this and results are…confusing. All I did was modify the default Worker project. I added a thread option and the ability to use a BinaryStream and Picture.FromData instead of Picture.Open. My test has 1071 png and jpg files to resize. Before each run I restarted the app and deleted the Resized Images folder. MBP is Bootcamped to run on the exact same hardware.
Results below:
Mac:
Threaded:
Picture: 196 seconds
BinaryStream: 200 seconds
Workers:
Picture: 274 seconds
BinaryStream: 182 seconds
Windows:
Threaded:
Picture: 124 seconds
BinaryStream: 205 seconds
Workers:
Picture: 115 seconds
BinaryStream: 106 seconds
I don’t think Workers are working (pun intended) as expected. I would expect the Workers to be considerably faster than using a thread and I’m not seeing it even though the fans on my MBP are running full blast with workers.
I’d love for others to examine Workers in greater detail to either confirm or dispute my results.
Just to be clear. The tests were with BUILT applications. Nothing was run in Debug.
While I can’t help, out of curiosity, how may workers are you spinning up and how many cores does the CPU have?
The worker is a console app.
So desktop app uses CoreGraphics for working with graphics, while Console uses a library called GD, which does a bit different things.
And I wonder whether Mac side does more on the pictures, e.g. color space correction.
Why don’t they use CoreGraphics?
I noticed a few years back that console framework graphics were significantly slower than desktop framework graphics.
We solved this by ditching Xojo pictures and using MBS & Einhuger plugins instead.
See the following for more info:
• MBS: PictureMBS / JPEGImporterMBS / PNGReaderMBS / TiffPictureMBS
• Einhuger: PictureEffectsRaw
Maximum core count is 4 and I set Core Percent to 90. The MBP is 2015 2.5 GHz quad-core Intel Core I7.
I did a comparison test with the worker settings of 3 cores and max 50% and time went from 274 to 545 seconds. It still kills me that the thread is faster than the Worker. I’ll have to try with the MBS classes to see how those affect this test.
I’m still think that Workers aren’t working like they’re supposed to.
One framework for console on all platforms. It has advantages.
More testing results. My conclusion is that the Picture Resize example is simply a bad example. Why? Because if you increase the number of pictures that the worker handles for each job to 20 you get a significant performance increase (at least for MacOS).
Mac Standard Picture.Open: To 196 from 274 seconds.
Mac Binary Stream: To 176 from 182
Windows Standard Picture.Open: No difference
Windows Binary Stream: No difference
This says to me that your Worker needs to be doing something really time consuming (i.e. not quick and quit) because there is a cost to starting and stopping a worker process.
So Mac, Linux and Windows all chose GD for drawings in Console?
They rarely agree with each others.
It’s choice of Xojo Inc.
Does each worker have to be validated by OSX every time it starts?
I honestly don’t know. Be nice to get a definitive answer from Xojo. There is some startup time so it’s not negligible.
I have a blog post coming tonight with more details on testing and observations.
It would be nice to know whether workers are launched new for each job or whether they stay running in background and wait for jobs.
macOS app validation should only be done once.