Controlling window of another application under Mojave

Targets macOS
1

I have a use case where I want one application to act as a window manager for another set of applications, and be able to either minimize or restore a window in those other applications. I can know either the process ID or window handle, but I’m having trouble finding anything that works under Mojave.

Conditions:

  • Internal use software; does not have to pass MAS requirements or even be notarized for that matter
  • Adding the window manager to the System Prefs > General > Privacy list is fine (or any similar settings)
  • Use of MBS or AppleScript or declares or whatever is fine too
  • Window titles are not necessarily unique
  • Multiple instances of the same GUI app name/bundle ID to control may/will be active
  • Each app to be controlled has a single document style window
  • The apps to be controlled are also Xojo based
  • I have been trying with the process ID, but if better to have the window handle I could get that too

To use AppleScript, I have not found a way to issue “tell …” commands to a specific process ID or window handle without using “tell application System Events” and “tell 1st item of (processes whose unix id = …) to …”. But Mojave seems to block any AppleScript to System Events.

To restore a window I have tried RemoteControlMBS.WinBringProcessToTop(process id) but it returns that it failed, and the process’s window is not restored. And nothing else in the RemoteControlMBS class seems to be relevant here.

I have also tried some things with AccessibilityMBS and ProcessMBS and CGSWindowListMBS, but not found the magic incantation yet.

I know I could do it by adding sockets to each program and setting up a network connection to pass commands. Or use Aloe Express or something. But it just seems like it shouldn’t be this hard to simply be able to request another app to minimize or restore a window. It seems the RemoteControlMBS or AccessibilityMBS class should be helpful here, but I haven’t figured out yet what calls to make.

2

To use Apple Script; you have to make each program scriptable. I think @Thomas Tempelmann has done this.

Otherwise you need a system of IPC, in order for one app to communicate with t’other.

There is another way; in that I know it’s possible, but I don’t know how. I use a utility called Moom to rearrange windows.

[quote=468400:@Douglas Handy]Mojave seems to block any AppleScript to System Events.
[/quote]
You need to give the controlling application the priviledge to do this; i forget what the option is called, but it’s in the Privacy settings of System Preferences.

Given all of Apple’s attention is on restricting functionality in the name of Security (I believe it’s a cover for basically turning the macOS into iOS); I would suggest that you put some thought into how you can do what you want without having this ability. I.e. I would expect this ability to go away in the future.

3

Yes, I understand it is necessary to add the controlling application to System Prefs > General > Privacy list of applications authorized to control other apps. And I may just give up and use IPC or Aloe Express or similar. Probably could have done that in less time then I spent trying to experiment with what I thought should have been simple.

Instead of Moom, I use Better Touch Tool, but that is part of what makes me realize this should have been possible once the app is added to the above list. And that does let me use the RemoteControlMBS class functions – I just don’t see one that would do what I want here.

However, as you point out, since I can control both sides, using another technique like IPC may be more future-proof.

Partly this was a learning exercise for something I thought would/should be simple. So far the lesson learned is that Apple makes some things way too hard.

4

The IPCSocket method works. I have a system built on a main application controlling a bunch (100+) of children applications and they communicate via IPCSocket.

Once you built the basic infrastructure to let the master and the slaves to communicate, you can define a protocol and basically send/receive whatever you want.

In short, the master will open a IPC socket at predefined path and Listen, while the slaves uses the same IPC socket path to Connect. Keep in mind, each time a slave disconnect, you have to Listen again from master.

You can have multiple sockets if you know already how many/which slaves you control to allow concurrent communication. You can even let the slaves to Listen and the master to Connect if you prefer. Only important thing is they agree on the same socket path.

5

IPC or UDP should be ok, depending on your needs. UDP should be faster.
However I’d be very careful about Apple changes. For a long(er) term solution I won’t look at Mojave only but also at Catalina.

6

IPC (and UDP too) works with Catalina (and Windows too). I’d avoid any solution involving AppleScript due to security complications, especially on Catalina.

7

Catalina has severely limited what scripting can do. Chances are you need to put some key in info.plist to authorize calls to system.events.

Some duckduckgo seems to provide some insight, though.

8

[quote=468415:@Massimo Valle]The IPCSocket method works. I have a system built on a main application controlling a bunch (100+) of children applications and they communicate via IPCSocket. …

In short, the master will open a IPC socket at predefined path and Listen, while the slaves uses the same IPC socket path to Connect. Keep in mind, each time a slave disconnect, you have to Listen again from master.[/quote]

Are each of your 100+ slaves using the same socket path? Or are you using a unique socket path for each slave? In my experiments, a single master/slave works as expected, but an additional attempt at either connect gets error 103 (NameResolutionError) and another attempt to listen doesn’t see the DataAvailable event.

I’d rather share a socket path if possible, more akin to UDP multicast. But while I have done UDP multicase/broadcast across a LAN numerous times, I have never done it for multiple apps on the same machine (localhost).

This Stackoverflow question makes it seem like you should be able to use UDP on localhost, provided you set the socket to allow reuse (which presumably I can do with OptionReuseAddressMBS and OptionReusePortMBS ) but I haven’t got that working either in multicast mode.

I can set a multicast group to “send to self”, but I have not found a way for the other apps to see the UDP datagrams too.

9

[quote=468500:@Douglas Handy]This Stackoverflow question makes it seem like you should be able to use UDP on localhost, provided you set the socket to allow reuse (which presumably I can do with OptionReuseAddressMBS and OptionReusePortMBS ) but I haven’t got that working either in multicast mode.

I can set a multicast group to “send to self”, but I have not found a way for the other apps to see the UDP datagrams too.[/quote]

For the sake of people looking at this thread in the future, it is in fact simple enough to send UDP datagrams between two processes on the same machine, using a UCPSocketMBS object and setting the .ReuseAddress and .ReusePort properties to true. MBS even has a sample application “UDPSocketMBS test” which is capable of doing this.

The same sample does not seem to allow multicast group or broadcast messages to be seen by multiple concurrent processes, though each instance reports it has joined the group and successfully sent the messages. So I’d end up needing a separate port for each client, and a way to manage those ports. At that point, it may in theory be potentially faster than using IPC sockets but in my particular use case I could come up with derived IPC path names that would be unique per client and not need to manage sharing of unique UDP ports between the master and slave.

10

Of course you need to have a socket for each application. Sorry if I was not clear on this.

11

I have similar system in place to have internal notifications between applications, using UDP multicast and works perfectly. Maybe you should check again. Will try to give you some code if find the time.

12

You mean UDP multicast within 127.0.0.1 for internal notifications? That is what I had not solved yet. I can do multicast UDP across the LAN, and now have UDP unicast to single secondary app on localhost. But not multicast yet, which is what I was trying to achieve.

Failing that, I guess what I could do is IPC where I use the process ID as a suffix in the path name. That would be consistent to derive in the path name in the other apps and stay unique among running processes.

13

For local UDP multicast you have to use a class D address. See: https://en.wikipedia.org/wiki/Multicast_address

You basically open a multicast UDP socket to an address like 224.0.0.1 and port at your choice. Use the MBS UDP class to have the reuse port option.

14

I had been trying the default class C address of 225.1.2.3 in the MBS demo. I’ll switch to class D and try again.

15

Please consider using UDP multicast vs IPC has an important difference: IPC is local to the machine, while UDP multicast is local to the LAN. Meaning other applications on the same LAN can join the group and receive, send messages.

16

In this particular case they are all on the same machine – while my UDP multicast experience in the past has been for other devices on the LAN.

In this case, they are more like helper apps, but single window GUI and not console apps, and running independent processes. What I am trying to make is an easy way to monitor all of those other somewhat related apps, and have an easy way to restore or minimize any given instance. Those other apps are periodically writing status information to named shared memory locations, which my monitor app reads and summaries into lists. But what I am trying to achieve is being able to chose one from the list and get its window to restore and be foreground, and minimize the window of what was previously viewed.

So it does not need to work across the LAN like normal UDP multicast. All I am really looking for is how to minimize/restore a window in another app. Which things like Moom or Better Touch Tool can do when given access in System Prefs.

Since I have not found the way to do that under Mojave, another approach will be to have a way to tell the app to perform its own restore or minimize. And IPC or UDP seem likely candidates. If I can get UDP multicast to work on localhost, then I don’t need multiple ports or IPC sockets and managing the share of port used etc. One advantage to IPC here is the helper apps could name their IPC path using the process ID as a suffix, and my monitor app would be able to derive what IPC path to use for that instance.

BTW, even using class D address of 224.0.0.1 and setting the reuse options on the UDPSocketMBS class, I was not able to get other instances to see the messages sent by each other. Each instance joined the group and could send messages, but only that instance reported the message in its own log window. So if you have an example of UDP multicast working for interprocess on 12.0.0.1 instead of the LAN, I’d love to see it.

17

Not sure if this is what you’re after, but I commonly control one of my apps with another (typically to plot results, do statistics, etc by app B on data generated by app A).

So app A would have a method like this:

[code] dim paramStr64 as string = EncodeBase64(echoStr,0)

dim targetBundleID as string = “com.pkstys.statsbuddy” ’ Stat Buddy’s bundle ID
dim eventClass as String = “aevt” ’ do we need this?
dim eventID as String = “sbcm”
dim cmdStr as string = “echo”
dim cmdParams as string = paramStr64

dim ae as New AppleEvent(eventClass, eventID, targetBundleID)
ae.Timeout = timeOutSeconds
ae.StringParam(“cmdStr”) = cmdStr
ae.StringParam(“cmdParams”) = cmdParams

if not ae.Send then
break
return false
end if

dim echoReturn as string = ae.ReplyString
return (echoReturn = echoStr)
[/code]

and app B would receive, decode and execute whatever command you send it in its HandleAppleEvent event handler. On the receiving end you can create a suite of command handlers to do whatever you want. For more complex commands I pass an XML-encoded dictionary which is decoded on the other end with any necessary fields that are needed. If you have multiple instances I suppose you could send the unique PID so only a single instance responds to a command sent to a common bundle ID?
This is all High Sierra BTW, but it seems to work on Mojave.

P.

18

Ahh, very interesting. This looks to be very workable, if I had only a single instance of each bundle ID running. I’ll have to test to see if multiple running instances of a bundle ID each receive the event, or only one of them. If they all do, then I can in fact include the desired PID as part of the passed parameter string.

Thanks for the idea.

19

Here it’s the code I extrapolated from my main class (UDPSocketMBS subclass) to handle a UDP multicast:

init the socket

// init and bind the socket self.ReuseAddress = true self.ReusePort = true self.MulticastLoop = true self.Bind(port) self.AddMembership(address)

send a message

call self.SendMessage( message, self.address, self.port )

receive the message in the DataAvailable event

[code]// read the data
dim d as DatagramMBS = self.Read

// if we have a datagram then parse the data received
if d <> nil then
// do something with the data
end if[/code]

that’s all and works perfectly even cross-platform.
Hope this can help.

20

Ahh, that made me realize what I was doing wrong in my tests. In the MBS demo, I was using the Join Group in each instance but not also doing the Bind button. With both Reuse… options and MulticastLoop set (which i was) and doing both the Bind and group membership (which I was not), all localhost instances were able to see UDP multicast messages. Even with a class C address.

Makes perfect sense in hindsight – without the Bind it was not also listening and only saw its own messages due to the MulticastLoop setting. Somehow I was thinking that by joining the group it would see the messages.

Sending a message to the multicast group makes each instance receive it once; sending to the broadcast address made each instance receive it twice (so I won’t do that ). But I can now make this work ideally regardless of the number of instances. And all sharing a single known multicast group and port.

Thanks!