Simple idea for a helper app framework

General
1

Xojo apps run on one core, threads or no, so it’s recommended that processes that can benefit from true multitasking run through helper apps.

I came up with an idea this morning that’s probably not unique to the world to run a helper app through a Shell. The idea is that the helper app would take a single parameter which could be as simple as a number or complex as Base64-encoded JSON, do its thing, then output a result to the shell. The calling app would use an Interactive Shell to start the process and return the results in an event.

I worked up a simple test and it seems to work exactly as expected. In Interactive mode, the Shell uses very little CPU time. Meanwhile, all my cores are put into play to process the task at hand. The nice thing is, the helper app can be stopped just by closing the shell, and when the app quits, all its helper apps go with it.

Am I missing a downside?

2

This would be better done via stdin to avoid any platform limitations on command line argument length.

3

I use a similar technique to interact with libraries that aren’t xojo thread friendly. It’s a heavyweight version of true preemptive threading, but it works for me.

4

Fair enough.

5

or pass IPCSocket path and connect form helper?

6

Getting large amounts of data back with the shell isn’t very fast. Also I’ve had problems assembling the data because it’s didn’t arrive in one piece. I’m going to look into IPCSockets for speeding up a part of my code soon.

7

This is true only in Mac OS X.

Using the shell for a helper is a normal way of proceeding, and playing on parameters and result is fine if you do not need more frequent interaction.

For big amounts of data, if speed is not an issue, creating a file is probably best.

I do like Xojo’s IPC, though.

8

I’ve written (and written an article about) this a few years ago.

I’ve found that most efficient way is to combine with a Shell and an IPCSocket:

The Shell controls the lifetime of the sub-processes. To kill a process, you can simple close the Shell. And if the process crashes, you’ll get an event in the Shell as well.

But the comms via the Shell i/o is slow. That’s where the IPCSocket comes into play.

9

[quote=266455:@Michel Bujardet]@Kem Tekinay The nice thing is, the helper app can be stopped just by closing the shell, and when the app quits, all its helper apps go with it.
This is true only in Mac OS X.[/quote]
Huh? Do you mean that on Windows and Linux you cannot control a shell process like that? I thought I could when I wrote my code back then, as I’m sure I’ve tested it on all platforms.

10

See here for my article: http://www.tempel.org/RB/MultiProcessing

1 Like
11

So my plan is workable if the data passed is relatively small, communication does not need to be interactive, and speed of passing parameters and results is not an issue?

Also, on Windows, the shell will have to be cut off manually, in the Shell’s Destructor I suppose.

Perhaps then the framework should automatically create the IPCSocket, establish communication with the helper, and pass the initial parameters that way. If the helper needs more information, it can raise an event in the host. I was also thinking about automatic serialization of classes so they can be passed back and forth.

(All theoretical since I have not played with IPCSocket yet.)

The helper app would need a class that represents the host, and the host a class that represents the helper.

(Thomas, I appreciate the link, and I’ll look at it to compare once I’m done. I find I end up with a richer product that way, even if I end up redoing parts.)

12

Communication can be interactive with an interactive shell.

13

On Windows, you can leak processes. They aren’t necessarily killed like on *nix systems, depending on what they are doing.

14

Sure, but the helper has to be interactive as well. In other words, there has to be some mechanism for the helper to get and send information to the host other than at the helper’s launch and when it quits. I’m thinking of a framework that has such a mechanism built-in.

15

In Windows the shell is not a child process. So terminating the app that called it does not terminate the shelled app.

16

You can perfectly send to the helper a instruction to terminate through IPC, or a declare to the terminate process.

17

Thanks Michel. I’ll play around with this once I get into it. Right now, this is just a bit of fun for me.

18

There is an example included with Xojo that demonstrate this technique as well:

Examples/Console/Multiprocessing

I’m sure Kem will come up with something more clever, though!

19

I have been playing with single helpers in OS X for a while. I usually use a plain Xojo shell and communicate with IPC. This is quite simple, really. I also used Valdemar De Sousa’s VDSC where a helper actually launches the main application upon system boot.

All that works very well.

I plan on using VDSC soon as well in Fonts Manager.

20

The unix design ethos of “lots of small atomic functions” is really useful once you get into it, and designing a stand-alone process that can be tested via the command line is very useful.

I’ve done similar things, but instead of using Shells I use xojo apps that are launched with command line parameters that tell it what IPCSocket path to use. In fact, a clever technique is to launch another instance of yourself (e.g. the same app that is running) in a “helper” mode.

I’m pretty sure there is a feature request to make these “helper apps” a first class Xojo feature.