Slowdown with 2018r1

Renaud_GOMBERT_WEBPO · April 19, 2018, 10:41am

Hi,

I have a benchmark code to check and compare evolution and differences between languages and versions, and I noticed a slowdown in 64bits release-compiled windows binary (have’nt checked 32bits yet) from 2017r3 to 2018r1.

The code is mostly this, where “number” is a window property and “n” the method argument (integer):

[code]For x = 1 To n
xx = xx
For y = x To n
yy = xx + yy
For z = y To n
If yy = z*z Then
number = number + 1
result = result + “[”+Str(x)+","+Str(y)+","+Str(z)+"]"
end
Next
Next
next

Return result[/code]
Suboptimal algorithm and long string concatenation are intentional.
I’m not complaining the thing is slow, but that it got slower with the latest release… more than 20 times slower with n=500 (about 800ms vs 30, both compiled with aggressive otpimization level).

Any idea of the possible cause?
Thanks

Greg_O_Lone · April 19, 2018, 11:27am

The compiler was upgraded to LLVM 6.

Renaud_GOMBERT_WEBPO · April 19, 2018, 11:59am

Thanks Greg, but IMHO that’s not an explanation by itself: Rust 1.25 also upgraded to LLVM 6 but none of my rust binaries got noticeably slower.

Kem_Tekinay · April 19, 2018, 12:28pm

You are compiling into a standalone app, not running through the ide, right?

Renaud_GOMBERT_WEBPO · April 19, 2018, 2:33pm

Yes Kem, standalone app (not debug) exactly as I did before when we had no other choice on Windows.

Kem_Tekinay · April 19, 2018, 2:35pm

Interesting. I have a copy of my M_Crypto app compiled with some older version of Xojo and when I compare the results of the stress test on Scrypt (an intentionally slow algorithm), the results are within a second of each other at around 21s.

Kem_Tekinay · April 19, 2018, 2:36pm

Oh, you said Windows and I missed it. My test was on the Mac.

Is this in a desktop app or command line? Assuming the former, what happens if you try the latter?

Renaud_GOMBERT_WEBPO · April 19, 2018, 3:50pm

Yes Kem, it’s a (Windows) desktop app.
Sadly I can’t check the command line, as I don’t have a Pro licence.
(I only purchased a Lite/Windows to evaluate the product)

The CL project is here if someone want to give it a try, or send me a compiled executable from both Xojo versions.

Thank you already to Greg and Kem for their kindness and willingness to help.

Greg_O_Lone · April 20, 2018, 3:48pm

Try the following pragma before the code, you’ll get a lot closer to the 2017r3 numbers:

#Pragma BackgroundTasks False

Renaud_GOMBERT_WEBPO · April 20, 2018, 4:10pm

Thanks a lot Greg, but no luck here… I already had this pragma (and a few others) set on both compilers.

#If Not DebugBuild #Pragma BackgroundTasks False #Pragma BoundsChecking False #Pragma AutoWaitCursor False #Pragma StackOverflowchecking False #Pragma NilObjectChecking False #EndIf

Renaud_GOMBERT_WEBPO · April 22, 2018, 6:18pm

So, nobody else on Windows noticed a slowdown of 64bit binaries from 2017r3 to 2018r1?

Could it be posible that some part of the new debug code just released for W64 is left in the binary even when debug is not checked?

ChristopheDV · April 22, 2018, 6:25pm

It should be faster regarding pure maths, string handling, … because of LLVM6. If I am not mistaken there was someone who posted benchmarks that showed it was much faster.

Regarding drawing speed, Windows builds are now very very slow compared to 2017R3. The flickering fix has a huge side effect on this. The mean reason why I do not build with 2018R1 for Windows.

Greg_O_Lone · April 22, 2018, 6:26pm

[quote=383741:@Renaud GOMBERT WEBPORTAGE]Thanks a lot Greg, but no luck here… I already had this pragma (and a few others) set on both compilers.

#If Not DebugBuild #Pragma BackgroundTasks False #Pragma BoundsChecking False #Pragma AutoWaitCursor False #Pragma StackOverflowchecking False #Pragma NilObjectChecking False #EndIf[/quote]
And theyre set in the method where youre having trouble?

ChristopheDV · April 22, 2018, 6:29pm

Found de benchmark:
It seems only string manipulation is faster.

2017R2:

Math (long integer) , 2000.0
Math (double precision) , 1355.0
Math (transcendental) , 1076.6
Math (scientific mix) , 1236.0
Math (composite score) , 1416.9

String Manipulation , 6176.8
Graphics (mixed) , 4876.0
Disk Read/Write , 885.6
Composite Benchmark , 2973.2

2018R1: (taken from one of the beta versions)

Math (long integer) , 2000.0
Math (double precision) , 1299.9
Math (transcendental) , 1076.8
Math (scientific mix) , 1235.4
Math (composite score) , 1403.0

String Manipulation , 7566.9
Graphics (mixed) , 5122.3
Disk Read/Write , 818.4
Composite Benchmark , 3335.8

Renaud_GOMBERT_WEBPO · April 22, 2018, 7:19pm

Yes the pragmas are in the “slower” method itself. But it’s not really relevant, since I run the same code anyway.

Timings are strange (here for n=500) :

2017r3 32bits debug : about 3-4 seconds
2017r3 32bits release : about 1-2 seconds
2017r3 64bits debug : N/A
2017r3 64bits release : about 0.04 seconds

2018r1 32bits debug : about 7-8 seconds
2018r1 32bits release : about 1-2 seconds
2018r1 64bits debug : about 7-8 seconds
2018r1 64bits release : about 0.8 seconds

Renaud_GOMBERT_WEBPO · April 22, 2018, 7:22pm

The “same” code writen in Rust (also using LLVM6) is run in about 30-40ms, the same time as the 2017r3 binary.

Renaud_GOMBERT_WEBPO · May 8, 2018, 12:27pm

Now that I purchased a Pro license I can test the command line version as well, and the result is the same.
The same code takes either 30-40ms or ~900ms depending on the release (both 64bits, Windows, aggessive optimizations).

Emile_Schwarz · May 8, 2018, 1:33pm

Same project, same OS, created and run with 2018r1 is way faster than run on 2015r1, same laptop.

The project load images and put them in a PDF on El Capitan

Go figure.

PS: on my first run under 2015r1 I said why is it so slow ?