Incremental compilation: Bug or not?

Do people think this a bug or expected behaviour?

I have a method that populates a Dictionary with values. It looks something like this:

mDict = New Dictionary mDict.Value(1) = SomeEnumeration mDict.Value(2) = SomeOtherEnumeration ...

This method is within a module. There are about 30,000 lines in this method, similar to above, just assigning values.

Without this method in my project, it compiles in about 2-3 seconds. With this method, it takes about 62 seconds. This is on an 8-core iMac Pro running Mojave. Even if change absolutely nothing and attempt to recompile, it still takes 62 seconds. This is using the LLVM (64-bit) compiler.

If I switch to Xojo’s old compiler (32-bit) then the project compiles in about 28 seconds on the first build. Like the LLVM compiler, it always takes 28 seconds to build even if I don’t touch the code in the module. I’m not bothered that the 32-bit compiler is faster (I know LLVM optimises). I’m just illustrating that both compilers do not seem to be incrementally compiling the module.

Can an engineer explain what’s going on here? I thought that both of Xojo’s compilers supported incremental compilation. Doesn’t that mean that if the source code within a module isn’t touched at all then it doesn’t need re-compiling?

Several people have made helpful suggestions on workarounds so I don’t need 30,000 lines of code in a method. For reasons that are not important - I need to use a Dictionary. I’ve tried an in-memory SQLite database and a few other suggestions but (once compiled) the Dictionary method is the fastest and is the most robust.

Just to be sure: Is this about “(Debug)Run” or a “Build”?
As far as I remember: Xojo always compiles everything when doing a Build. Incremental compilation is only for DebugRun’s (except for the 1st one after loading a project of course).

You have increment build switched on? And compilation on default, moderate or agressive?

To clarify, I am seeing the long times when doing a debug run.

Compilation is set to default.

There is no setting to enable incremental building is there?

a debug run may NOT in fact be recompiling the module
but it does take some time to link together everything
that may be the 28 seconds

[quote=456325:@Garry Pettet]To clarify, I am seeing the long times when doing a debug run.

Compilation is set to default.

There is no setting to enable incremental building is there?[/quote]

Auto Increment Version, is only working for builds as far as i know.
just as norman says ther must be a minimum.

The lowest i’ve seen in time is 9 seconds (debug run) on a high power machine (a full GUI application)

there are ways to see what is / isnt being recompiled

its one of those many temporary files xojo creates

first thing you need is the PID of the specific instance of xojo you’re interested in
it makes life simpler IF you just run one version at a time
once you start Xojo you can look in Activity Monitor & get its PID

If you find the dir referred to by SpecialFolder.Temporary
a two liner in Xojo like this would suffice

   dim f as folderitem = SpecialFolder.Temporary
   break

for me on 10.14.6 this is buried in

/private/var/folders/3t/wyqsbmsj4bbf_z_wdrjnp0zw0000gn/T/TemporaryItems

inside there should be a dir named "xojo scratch "

inside there will be different dirs for each open project that you have saved & run
they are generally named in a way that its not so hard to tell which is which

if you open this in the finder then you can sort by last modified
every time you run you can see the modification dates of things and tell what is / isnt being recompiled

Just tested it in a simple test project, took aaaaages to finish up and run, an array of similar size took no time… I’d say bug

Sorry to be ‘that guy’ but wouldn’t a SQLLite database be a better solution for that?

Like I said Jeff, I’ve run tests. 100,000 characters tokenised using a dictionary takes 36,000 microseconds. Using an in memory SQLite DB it’s >1,000,000 microseconds.

[quote=456362:@Derk Jochems]Auto Increment Version, is only working for builds as far as i know.
just as norman says ther must be a minimum.

The lowest i’ve seen in time is 9 seconds (debug run) on a high power machine (a full GUI application)[/quote]
Auto increment has nothing to do with incremental compilation - it just increments the build number.

I’ve done tests. Without the mammoth method the whole project takes 3 seconds to debug and run. With the mammoth method un-commented, it’s 60 seconds on my iMac Pro.

that may be link time and not compile time (although the distinction is kind of esoteric)

Could you not store the data in a SQLite database on disk and read that into a dictionary at startup?

In a project of mine I use pairs (strings). Something like:

redim arrayPair(SomeEnumeration.ubound)
for i as integer = 0 to SomeEnumeration.ubound
arrayPair(i) = SomeEnumeration(i) : SomeOtherEnumeration(i)
next

and I’m rather satisfied with it (about 40000 instances); and accessing the values “seems” quicker than an in-memory SQLite DB.

So I’ve managed to solve this. What I did was create a separate CLI tool to parse the UnicodeData.txt file and reformat it into lines where each line is a codepoint:category pair. I then stuffed that big string into a constant in the module and at initialisation time I split that string, iterate over it and put it in the dictionary. The project is back to compiling in a couple of seconds and the result is as fast as ever.

Thanks for all the insights.