Workers nightmare

Hello guys,

It would help if I could get some clarifications for the workers side .

Apparently the code works well in the debug, then once I build the app the app stops working .

I assume that in debug mode it simulates the workers while on the build app it actually creates helpers.

Now, I do have an app that processes some files, the first part is ok, it works, where it processes the files.
The second part should connect to a SQLite Database and update some records , now, when reaching to this part the app stops, on some system debug logs I get that the app cannot connect to DB even if I included the DB class and modules in the project . I try to initiate the DB separate in the worker and it keeps on looping like crazy and does not work at all.

Seeing the docs and the examples , I guess you can pass only some data over the worker as string and that’s about it, and only one time , so how do you manage multiple data processing ?

And as a side note @XOJO Team, it would be nice to have a Search on the top of the examples in XOJO, the new structure is a nightmare until you find something there.

Thanks.

You need to open your SQLite Database as multi-user WAL, so your main app and your Workers can all access it at the same time. By default a SQLite Database is single user only.

2 Likes

Already done that , this is not the case .

I started to put debug logs everywhere , so the code initiates the database then DB File becomes Nil so I assume that there is something fishy with the “Project Items To Include” part .

Perhaps something to do with permissions? (i.e. worker app processes when spawned from built app don’t have permission to directory where SQLite database resides)

I think I found the problem, Apparently it does not like to have all the db in a global module that is of course embedded as well in the worker so I ended up moving all the code from Module to a class and now all works. I guess I’ll have duplicate code all over for each worker but as long as it does the job it should not be a problem.

Thanks.

When debugging Workers are threads, when building they are actually separate applications included within the main “package”.

So you need to make sure that each worker creates it’s own instance to a database.

2 Likes

@Xojo It seems that Workers are still not working properly in 2022R3.2 I did refactored my code in order to optimise little bit the app and now i get tons of errors that Modules apparently are not added or don’t exist in Workers.

If i add the Documentation line

List of Project Items to include in the Worker Console app. No classes or modules are included by default. Enter one project item per line. To include all classes and modules, enter IncludeAllClassesAndModules .

Then i get an error saying that

ImportSource.Name
Project Item ‘IncludeAllClassesAndModules’ in ProjectItemsToInclude could not be copied because it does not exist.
ImportSource

So it would be nice to have this sorted in the next release maybe as again i need to waste time to redo all the code and try to have it work, when it was supposed to work.

Added this as issue https://tracker.xojo.com/xojoinc/xojo/-/issues/71021

If you need more details let me know.

Thanks

Robin will ask for an example project anyways. Could you make and add it to the issue?

Why would you add all classes and modules to the worker? Just asking for the use case. That would make the worker really big. Do the classes need to be console safe?

Why don’t include all modules and classes automatically that are used in the worker’s code? This manually include thing makes using a Worker really complicated and error prone in larger projects. Xojo has all the infos to include, so…

1 Like

Because that would be very prone to error. If you have two similar class names and intended to use one but typed the other by mistake it would never spot your error. Surely it is better for you to declare your intent and be warned when something doesn’t meet expectations.

1 Like

I see that completely different. To detect such errors, we have the debug build.

That doesn’t so much detect errors as allow you to see what is going wrong when it does. You would have to test everything very closely to know it was going wrong in the first place.

It’s a bit like saying you don’t want to have to declare variables because you can use the debugger to work out that you have a typo in your code using the debugger.

An analyzer checking propagation of dependencies could fill such list with proper and correct data under request.

2 Likes

While it sounds simple, the reality is that you’re talking about a chicken and egg problem. “knowing” everything that’s used in a project requires a successful compile and compiling the project successfully requires only having console-safe components. The place where it gets trickiest is with subclasses, because they’re not explicitly tagged as being console or not.

Besides, to keep the workers as small as possible, being explicit is going to give you the best result.

Again, Xojo (the compiler) has in the code all the information it needs, as Rick has already noted. It’s more a matter of will Xojo provide such a tool to automatically create the dependencies for the worker, or do it silently in the compiler, for example. I’m not saying this is “easy”, but it should be doable.

No, that’s a totally different thing. It’s just that Xojo does the work for us to detect the existing dependencies, which is what happens in the debug build but in a different context (desktop <> console).

We will have to agree to disagree about that.

Yes :slight_smile:

BTW, errors along these lines will not display in the debugger, where Xojo is just using threads to simulate. In my experience, they will only throw when trying to build.

You’re just not thinking this all the way through.

The compiler isn’t involved at the point when the binary for the workers is being split off from the main project. Basically, the IDE creates a second worker-only project, compiles the worker, and then compiles the main project and incorporates the worker binary in the build.