Benchmarking improvements in Xojo 2023r4

General
1

A few weeks ago, I had a blog post about Performance improvements in Xojo. Now that Xojo 2023r4 version is available, we got a few benchmarks to verify if things are faster.

Stack Overflow Check

Let’s test the improved stack overflow check. The test application basically performs a few loops to run about one million function calls. Each function call performs a stack check, so doing a million stack checks takes some time. Our test shows this improvement for StackOverflowChecking:

Seconds Where Xojo Version
57.876 Debugger Xojo 2022r4
31.566 Built App Xojo 2022r4
19.456 Debugger Xojo 2023r4
0.620 Built App Xojo 2023r4

You notice that the debugger is now running the app faster than it used to be in the built app.

Main Thread Check

Then we check the improvement of the main thread check in control getters. Whenever you access a control method or property, Xojo will check for whether you are in the main thread. This check got improved in the new version. Here are the result for our UITrap performance check:

Seconds Where Xojo Version
99.839 Debugger Xojo 2022r4
70.906 Built App Xojo 2022r4
25.665 Debugger Xojo 2023r4
6.065 Built App Xojo 2023r4

CLong function

Now let’s check if CLong got faster. One of the many utility functions to convert from string to integer:

Seconds Where Xojo Version
70.290 Debugger Xojo 2022r4
37.520 Built App Xojo 2022r4
4.500 Debugger Xojo 2023r4
1.407 Built App Xojo 2023r4

IsEmpty function

While we learnt that checking with = “” is faster, some users still prefer calling IsEmpty for better code readability. Since this function was improved from checking byte count instead of counting characters, it is now much faster. Let’s check:

Seconds Where Xojo Version
113.888 Debugger Xojo 2022r4
33.636 Built App Xojo 2022r4
23.835 Debugger Xojo 2023r4
0.815 Built App Xojo 2023r4

BackgroundTasks

And now we can test with BackgroundTasks on to see how much that takes. Each loop calls internally a function to check if scheduler needs to switch to a different thread. This functions got optimized, so it is now faster.

Seconds Where Xojo Version
13.801 Debugger Xojo 2022r4
1.128 Built App Xojo 2022r4
9.919 Debugger Xojo 2023r4
0.583 Built App Xojo 2023r4

All test made on a MacBook Pro with M2 and with default optimization level.

Real world?

The optimization is great news for all Xojo users. The IDE is snappier, the built applications work better. Your milage may vary, but especially web projects with their framework in Xojo call a ton of methods and there you benefit a lot here.

Does it affect your application? Maybe. If your application is mostly waiting for user input, network packets or disk I/O, then you may not notice it. But if you have loops doing some processing of data, you may see a benefit.

Please try the new version and let us know how well it works. Of course we keep an eye open for more potential optimizations.

19 Likes
2

I wonder how mich faster the IDE itself is built now since this change? I can remeber that it’s been said that builing the IDE normally takes a really long time…

3

William made a blog post about this: Small and Simple Changes to Speed Up the Xojo IDE

1 Like
4

I meant the actual building of the IDE itself, this blog is about icon drawing speed in the ide.
Xojo is made in Xojo, and i’m interested in how many % of speed the build chain is faster for the (huge sized) IDE itself ?

5

The IDE is snappier, especially if you have several tabs and windows with a lot of controls.
Launching Xojo with a lot of plugins is faster too. Same for loading (large) projects, that too is faster.
Typing is also faster.

That said, compiling apps is not faster (it’s exactly the same).

6

Because the compiler is an independent and already optimized C++ code (LLVM is) with its performance not influenced by Xojo optimizations.

But… Built apps are.

7

I did make a request to Reevaluate the LLVM optimization features

I assume a few optimizations take a long time, but have almost no effect.
While others are not enabled, that have a big impact.

6 Likes
8

I like the approach:

Fast build (for debug compiling/recompiling as fast as you can)
Normal (some good optimizations, default)
Optimized (more optimizations, takes more time)

Logically all those options must be safe, without side effects (all apps should run correctly).
I do remember that Xojo optimized code in the first editions had side effects, like solving wrongly complex “IFs” that worked ok in normal mode. So I never opted for optimized after it… years ago. Maybe some bug already corrected but I can’t afford errors.

1 Like
9

I have the weird phenomenon that an external declare into a custom NSAlert (runModal) that is then translated by a Xojo method before full returning will not return a valid result anymore once I switch to more than standard optimisation. I did find a workaround by removing that translation method but have no idea why working code is optimised away …

10

Some optimizations tries things as reading variables into CPU registers and caching things and reusing them instead of fetching its value again and again and some other tricks like removing parts that the compiler thinks are dead code / not used. Some of those data, the compiler passes, but the linker removes. Some of those things may be risky.

You luckily found something not failing silently.

A workaround could be a special #Pragma to avoid some of those optimizations like code/data striping don’t act in some blocks/variables/properties.

You should declare your NSAlert and data with this “don’t optimize/strip” pragma

No sure If William will have all tools to make this happen.

11

I like the idea of some new Pragmas. Personally I’d love something like:

#Pragma Inline SomeMethodCall()

to tell the compiler to just copy the method contents to the location of the pragma.

2 Likes
12

Ulrich should report his problem and someone from Xojo Inc. should check it.
If optimization removes the wrong thing, that is a problem.

13

Well, it does. It is its nature. Remove things it thinks is not used.

The problem is “what is a wrong thing?”, “what is not really used?”

Sometimes I think that the user must hint the compiler “don’t touch this”, no one is using it here, but an external lib that I’ll call will do it.

And sometimes you don’t care if it removes something because you agree too.

14

He did:
https://tracker.xojo.com/xojoinc/xojo/-/issues/75169

Thank you for your help, @Christian_Schmitz!

15

For this specific issue, Christian found that the external declare will work if its result type is set to Int32, which is IMHO not the right data type per Apple’s docs. But anyway, a much better workaround than what I found.

16

Seems wrong in some unusual way.

Because NSInteger should be Int64 if your app was 64bit.

Seems like taking 1/2 of the value because the other half contains “dirt” and should not.

17

That’s how I see it too.
And as mentioned, it works with Integer and standard optimisation.

18

Looks to me like the Xojo compiler does output a different code for Int32 compared to Int64, which then isn’t broken with optimization.

Like they probably do a bitwise And on the result to remove the top bits or similar.

Whether Int32 or Int64, they both are stored in a 64-bit register and for the NSAlert function, you don’t need the higher bits.

19

Well, it’s broken. How broken only deep inspection will find out.

A fix is needed, Ulrich is safe for this one. Others may not, because discarding the top 32 bit of a 64bit value usually does not end well.

antares-orb-3-rocket-explosion

1 Like
20

There’s a feature request for macros, they would have been really useful in helping me bridge API 1.0 and API 2.0 in OAK.

1 Like