Has anybody benchmarked XOJO against other Basic Compilers ?

Windows 7 VM with 4Gb RAM under 2014r1.1 I get 18.32 in debug & 2.1 compiled
I just shoved Dave’s code in the open event of the default window in a new project

2.1s on my Win7 VM, 3.9s on an old 1.6 GHz Core 2 Duo PC notebook I keep around for testing.

[quote=85640:@Gary Miller]I like the IDE that XOJO provides but have been unable to find any benchmarks comparing the speed of generated XOJO code to that of the the other BASIC compilers or other languages.

My application currently has a 26 second response time in VB.Net and I’d like to cut that in half if possible.[/quote]

Software design usually trumps CPU / compiler, and can do so by an order of magnitude or more. If you want to use Xojo for some specific reason (IDE, cross platform) then port and optimize. If you’re just looking for speed, optimize your code first and see what you can get. The optimizations for Xojo may be different than for VB.NET, and it’s anyone’s guess as to which will be faster when comparing the fastest possible code in each.

There are some tasks where language and compiler weigh more heavily, and Xojo is generally not as fast as C++ or some of the other BASIC compilers in these categories. C has language constructs for manipulating arrays that trump the equivalent code in Xojo and most (any?) versions of BASIC. C and some of the other BASIC compilers are faster at math heavy code. That said, XojoScript uses LVM and…based on my limited testing…seems comparable to the other BASIC compilers on math heavy code. (Someday Xojo will use LVM for compiled apps.)

There’s no support for preemptive threads in Xojo which are necessary to take advantage of multiple cores from a single process. Another solution is to launch multiple helper apps which the OS will schedule on multiple cores. This can work very well and is actually easier to debug then an app using preemptive threads.

I get about 3.1 seconds on an old windows 7 laptop: Pentium 2.2Ghz, 4GB of ram.

But this sort of benchmark is pretty meaningless with respect to OP’s issue. To know for sure if Xojo will produce faster binaries he’d have to port it to Xojo and see.

I’m betting that it can be made to be faster than the VB version.

Without knowing what your program does during these 26 seconds, it is impossible to predict what another compiler may do. To cut the response time, you may want to start by optimizing your code in VB first. Changing development tool may ultimately shorten execution, but will have taken more than seconds to port the program and learn the language. Whatever it is, Xojo, FreeBasic or anything else…

On My Mac… Compiled code with #pragmas in Xojo

Cocoa -XCode 3.65 secs
Cocoa -Xojo 4.00 secs
Carbon -Xojo 4.08 secs

Windows 8.1

without disablebackroundtasks 10 secs
with disablebackgroundtasks 2 secs

not sluggish at all

I really have no time to make a test now, but what about XojoScript running this code?

I am pretty curious if there is a big difference and LLVM does the work!

Just ran Dave’s code on my Windows 8.1, Xojo 2014 r1.1 machine:

17.4 seconds in IDE
2.4 seconds compiled

20.8 seconds IDE
2.3 seconds compiled

31 seconds IDE
2.6 seconds compiled

OS X 10.8.5 - Xojo 2013r3.3
Mac Pro 2x2.8 QC Xeon
8GB RAM.

43 seconds IDE
3.1 seconds compiled

OS X 10.6 - last version of RB
iMac (Early 2008)
3 GB RAM


18.4 seconds IDE
2.1 seconds compiled

OS X 10.9 - Xojo 2014
MacBook Pro 17" (late 2011, last model sold by Apple in summer 2012)
8 GB RAM

XojoScript :
0.35 seconds to 0.9 seconds IDE or compiled

OS X 10.9.3
iMac 2.5 Ghz Intel Core i5
4 GB RAM

Gives some perspectives to optimise long running operations, does it not ? :slight_smile: Placing the same empty loop in a XojoScript shortens its execution time compiled by 4… So this is LLVM…

Instead of entering painfully the source as “this+endofLine+that” and so on, as described in the LR, I typed the source in a text file I added to the project and then used as a string.

Note : For the sake of curiosity, I timed a 1 second timer with microseconds. I get results of 1.59 to 1.86 microseconds/1,000,000. So the number of seconds for the loop is probably exaggerated by 50%. Or timers are not as precise as expected…
Also, the results for the empty loop where not stable, and once in a while went down to 0.12, and up to 2.1. I had expected steadier response times, but this is how it goes… A more thorough benchmark probably calls for a series of measurements and graph…

[quote=85934:@Michel Bujardet]XojoScript :
0.5 seconds to 0.9 seconds IDE or compiled[/quote]

With BackgroundTasks False (in the script) and Precompile(XojoScript.OptimizationLevels.High) I’m getting ludicrous speed even in the IDE: 0.0000518 seconds (literally under 52 microseconds).

At first I thought it had skipped the loop entirely. But adding x = x + 1 to the loop and calling Print(str(x)) gives the correct output.

Then I thought that LLVM must be recognizing the code and compiling a shortcut. So I modified my loop to try some real but useless math for a million loops. XojoScript was about 30x faster then running the code in Xojo.

This is the kind of thing I’ve seen before with certain narrow…but potentially important…categories of code. I can’t wait for the LLVM compiler upgrade.

Timers do not fire with reliable precision. I wouldn’t judge Microseconds by a Timer.

I was more suspecting the timer of being inaccurate :wink:

[quote=85945:@Daniel Taylor]With BackgroundTasks False (in the script) and Precompile(XojoScript.OptimizationLevels.High) I’m getting ludicrous speed even in the IDE: 0.0000518 seconds (literally under 52 microseconds).

At first I thought it had skipped the loop entirely. But adding x = x + 1 to the loop and calling Print(str(x)) gives the correct output.
[/quote]

As I wrote in a previous post, I still think empty loops are not really a pertinent benchmark. Real programs contain much more than math, with strings, graphics, different types of logic and variables, which create delays of all sorts. The OP never reappeared so we may never know what his program does in 26 seconds. If he does disk or network access, for instance, even supersonic data processing will be taken to a crawl anyway.

But fact remains, XojoScript IS FAST !

I have an Intel i7 8GM with a 256GB SSD.

The code reads a a main.nlp that has includes that contain 46,500 complex patterns (and growing) at 10 lines each on average.

They are parsed into an in memory array of structures and a QuikSort is used sort the patterns into search priority order.

That takes 26 seconds.

The user enters an input. And the input is matched against the 46,500 patterns.

The pattern matcher is much more expressive than a regular expression pattern language.

Patterns can have subpatterns that are stored in pattern variables such as

[Male_First_Name]="{Aaron|Abe|Abner}"

And can extend for several hundred entries.

So there is a lot of string searching, slicing and dicing going on. No complicated math, mostly just natural language processing (NLP).

The pattern matcher is recursive.

With optionals (optional1|optional2|optionaln} parenthetical delimiters
With required {required1|required2|requiredN} curly brace delimiter

Patterns can contain nest optional, required and pattern variables.

When a pattern is matched a Template associated with that pattern is executed.

The templates execute quickly so speed is not an issue in this part.

Since many patterns can start with the same substring such as …

<< Where{(’)s| is} (the )(city of )City( at)(?)
<< Where{(’)s| is} (the )(country of )Country( at)(?)
<< Where{(’)s| is} the (state of )State( at)(?)

Many of the patterns are partially matched until a full match is found.

If a user input does not match any of the patterns then it has attempted to match all 46,500 patterns.

That should give you a feel for the amount of string handling and parsing that is going on.

Thanks very much for all the attention this question has generated and all of the good information!

Them’s fightin’ words! :slight_smile:

Seriously, I’m not sure what that means. Can you rephrase or give me an example?

[quote=86188:@Gary Miller]So there is a lot of string searching, slicing and dicing going on. No complicated math, mostly just natural language processing (NLP).

The pattern matcher is recursive.

With optionals (optional1|optional2|optionaln} parenthetical delimiters
With required {required1|required2|requiredN} curly brace delimiter

Patterns can contain nest optional, required and pattern variables.
[/quote]

At first glance, parsing the request is probably fast, then what takes a long time is the amount of INSTR() going on to power the matcher, and the search within 46,500 patterns. So, no disk access.

My first reaction to such a project is the use of indexes in the database, and RegEx in the matcher to speed up processing.

Maybe a good benchmark would involve generating 46,500 random strings with for each five instr() search.

I have built a small app with a ProgressBar, a button and a field for the result. Here is the code in the button :

#pragma DisableBackgroundTasks dim that as integer dim buff as string dim ii as integer dim starter as double = microseconds/1000000 ProgressBar1.Maximum = 46500 ProgressBar1.Value = 0 for recordnumber as integer = 1 to 46500 Progressbar1.Value = recordnumber buff = "" for stringlength as integer = 1 to 1000 ii = App.Randomizer.InRange(32, 223) buff = buff+chr(32+ii) next stringlength that = instr(buff,"Onceuponatime") that = instr(buff,"Aprincesswhas") that = instr(buff,"asleepinacasttle") that = instr(buff,"shehadbeencursed") that = instr(buff,"byabadapple") next recordnumber TextField1.Text = str((Microseconds/1000000)-starter)

Sub Open() //Randomizer is an app property as random Randomizer = New Random End Sub

in the IDE or built execution takes 46 seconds, 44 when the ProgressBar is out.

Of course this test does not mean much, but it does show that instr() takes time :wink:

OS X 10.9.3
iMac 2.5 Ghz Intel Core i5
4 GB RAM