I finally had time to make a Xcode project to show the problem to Apple. The problem shows easily after loading some dylibs and has nothing to do with Xojo.
FB11721327 for anyone at Apple.
Screenshots like this:
you see crashes after 385, 442 and 506 dylibs.
I uploaded the sample project and the dylibs to Apple for testing.
And no plugin code is executed to make this fail.
We have conflicts of information here.
Not really.
The crash happens with a small probability on loading a dylib, maybe 1%.
Xojo’s internal plugins have 18 dylibs.
If you have database plugins, that may be another 5 dylibs.
MBS Plugin adds over 500 dylibs with all plugins installed.
The probability is small and you see it easily with all our plugins. But you may also see it with Xojo alone, but not as often.
@Christian_Schmitz : did you create a DTS? Otherwise, the bug will not be fixed in a timely manner.
How many dylibs do other apps load? There would be a small chance that other apps would crash. And I’ve never seen that. I still think that Sam is correct and Xojo does something that macOS doesn’t like.
On my systems it always crashes. 100%
And maybe (not) important I check several apps that also load a lot of dylibs and they do not crash.
I did create a ticket for DTS on this.
Well Christian did reproduce it in a non Xojo App…
Which sort of eliminates the chance that Xojo is doing something wrong.
Well, Xojo Inc. could change the way they load plugins.
e.g. have a helper load them in background and maybe one helper per plugin. Send the details for the plugin out as JSON to the IDE, so the IDE doesn’t need to load them all.
Cache the information per plugin per file hash, so you only redo it if the plugin changes and keep multiple of them per plugin for different date to avoid redoing it when switching between different sets.
And then you just have new layer of problems that will always brake.
Not to mention Control plugins which need to actually run in the IDE and could not run from a “worker”. And then there is the case of non control plugins which might be dependency for Control plugin which also might need to run in the IDE.
Who knew that macOS had its own DLL Hell, lol
Yes, the IDE may still need to load control plugins.
There is no easy solution beside Apple fixing their loader.
Only way to make it possible would be to have plugins marked with metadata or different file ending marking them as safe to load in worker.
Since the IDE would have no way to know if it contains dependent class for control, or would not even have way to know if plugin contains a control before actually opening it up.
I’d be happy to include some metadata in the archive to tell the IDE whether the plugin part has controls.
That may be something like 20 of 500 dylibs with controls.
Funny, I know a guy who tried to rally support for a plugin metadata file and got nowhere…
Different file ending might even be more beneficial so that IDE can know even before trying to open the archive. = More load offloaded.
I think we just need what we have today + dependency file + new organization
Every PluginFile .xojo_plugin should need a companion PluginFile .xojo_plugin_dep (dependencies)
Those xojo_plugin_dep files only has a list of dependent plugins that should be loaded before it, so the contents of PluginFile.xojo_plugin_dep could be empty for self contained or for example
core_funcs
aux_funcs
so we know that this plugin PluginFile.xojo_plugin only can be loaded after core_funcs.xojo_plugin and aux_funcs.xojo_plugin because it depends on them.
So, now the loading routine would be
Shared classes used in many places could be grouped in some “aux plugin” and all plugins needing them would just list them as a dependency.
The new system would not read *.xojo_plugin anymore. Only *.xojo_plugin_dep, and the resulting ordered list will be loaded. So putting a zzz.xojo_plugin in the plugins folder has no effect without a companion zzz.xojo_plugin_dep that will include it in the list
A compiled app should obey the same tree of dependencies, but now taking in consideration only the used content, and respect such loading order.
Excellent work @Christian_Schmitz - if you notice, all 3 crashes are different, happening in different functions with different belhaviors. Feels a lot like random memory corruption, no?
I think this sort of crash could reasonably lead to a securty issue, such as arbitrary code execution. You might want to report it to Report a security or privacy vulnerability - Apple Support
In my expericence, Apple responrds to security bug reports very quickly.
The order or dependency does not really matter any more since Xojo reads them all now and then later evaluates and resolves the types.
(this was not always the case)
I was afraid of some out of order activation causing unsolved symbols ending on crashes. Or duplicated symbols that could be externalized once and shared. If that’s not the case… Let’s just anxiously wait the Apple response.
Hopefully the answer is not “don’t load so many dylibs”