Need help with Cocoa Declare

I have limited experience with Cocoa declares and need help constructing one calling NSAccessibilityValueChangedNotification.

I need to call this after dynamically in code swapping over my main window’s menubar so that Cocoa’s accessibility inspector registers the change and updates the available UI elements list for dictation commands. Right now without this Cocoa doesn’t register the change unless the app loses focus to another app and then regains focus.

https://developer.apple.com/reference/appkit/nsaccessibilityvaluechangednotification

void NSAccessibilityPostNotification(id element, NSString *notification);

I’m not sure if I have to pass Ptr(window.handle) as the object id or Ptr(window.menubar.handle) as the object id or create a reference to the object in another way.

I also don’t understand how to pass the NSAccessibilityValueChangedNotification parameter. Should this be a class or constant value?

Please can someone help me out with this. Many thanks.

Without guarantee (untested):

Declare Sub NSAccessibilityPostNotification Lib "AppKit.framework" (id as ptr, notification as CFStringRef) NSAccessibilityPostNotification(ptr(control.handle), "NSAccessibilityValueChangedNotification")

or id as integer and then pass only control.handle and the Notification name.

The control depends on what control you want to send the notification.

EDIT: Tested now and got no errors.

Hi Ulrich. Thanks for the code. It makes sense when I see it. Unfortunately I have tried all the variations below (separately of course) and while it compiles it doesn’t seem to update the Cocoa dictation Speech Commands window. Any ideas why this might be?

[code]Declare Sub NSAccessibilityPostNotification Lib “AppKit.framework” (id as Integer, notification as CFStringRef)
'Declare Sub NSAccessibilityPostNotification Lib “AppKit.framework” (id as ptr, notification as CFStringRef)

NSAccessibilityPostNotification(myMenuBar.Handle(MenuItem.HandleType.CocoaNSMenuItem), NSAccessibilityValueChangedNotification")

NSAccessibilityPostNotification(myWindow.Handle, “NSAccessibilityValueChangedNotification”)

NSAccessibilityPostNotification(myWindow.MenuBar.Handle(MenuItem.HandleType.CocoaNSMenuItem), NSAccessibilityValueChangedNotification")

NSAccessibilityPostNotification(Ptr(myWindow.MenuBar.Handle(MenuItem.HandleType.CocoaNSMenuItem)), NSAccessibilityValueChangedNotification")[/code]

I’m not sure because I never explored Accessibility. By the reference the notification is sent to any object that registered to receive the notification. Did you do so (or is there any reference that says it would automatically)?

I’m not sure what you mean. I’m swapping the main window’s menubar over with another menubar (in code) but the other one has not appeared before in the running app so are you saying that maybe I need to use another accessibility declare first to perhaps register it and then send the change notification?

I can only refer to the small bit I’ve read, and from what I see the code

To me this sounds like the speech commands window should register to the control from which you send the notification. Usually notifications are sent via a NSNotificationCenter, and it sounds like this would be the case here too. But again, I am not sure about all this accessibility stuff. Hopefully someone else here has more knowledge about this than me.

You’re using the wrong name for the notification. These constants are not always using the same values are their names.

To get the actual name, use XCode. Make a new Cocoa project, and then will in code like this into the file “AppDelegate.m”, inside the - (void)applicationDidFinishLaunching:(NSNotification *)aNotification function:

id s = NSAccessibilityValueChangedNotification; NSLog (@"%@", s);
Run it. In the console output panel, you’ll see the actual name. In this case it’s: AXValueChanged
So, pass that as the 2nd parm to NSAccessibilityPostNotification.

I think it means the speech commands app has registered with the OS to receive notifications when any app’s UI elements change so it may not be necessary to specifically register a native control like a menubar, only a custom one.

And furthermore, standard UI updates should automatically send notifications. But for whatever reason swapping over menu bars in code doesn’t seem to send the notification and the ones I have sent don’t seem to work.

I found these links which have a little more explanation but I can’t really make head or tail of it:

https://github.com/ralcr/hxcocoa/blob/master/osx/appkit/NSAccessibility.h
http://stpeterandpaul.ca/tiger/documentation/Accessibility/Conceptual/AccessibilityMacOSX/OSXAXModel/chapter_4_section_4.html
https://developer.apple.com/library/content/documentation/Accessibility/Conceptual/AccessibilityMacOSX/OSXAXmodel.html
https://developer.apple.com/reference/appkit/nsaccessibility

Thanks for your help so far and as you say, hopefully someone else might be able to advise on what I am doing wrong.

You can do it easier:
Just us a Swift playground and paste “NSAccessibilityValueChangedNotification”. It will show "“AXValueChanged”.

D’oh! If this isn’t a clear sign that I did not work on AppleLib for a long time … Or the advent of me gaining another 0, age-wise?
Thanks for reminding me, Thomas!

Thanks. I tried both versions below but unfortunately neither worked.

[code]NSAccessibilityPostNotification(myWindow.MenuBar.Handle(MenuItem.HandleType.CocoaNSMenuItem), “AXValueChanged”)

NSAccessibilityPostNotification(Ptr(myWindow.MenuBar.Handle(MenuItem.HandleType.CocoaNSMenuItem)), “AXValueChanged”)[/code]

It usually helps to start with a working example. Have you got any swift or ObjC examples that show that passing the menubar’s object reference is the right way to go?

Also, if you could verify that it works with other controls first, such as for a textfield, that would help figuring out if we’d gotten it right in general.

I don’t actually think it has anything to do with swapping menu bars over. Right now I’m trying it in a blank test project and dynamically adding a new top level menu item to the menubar to see if it registers it but it doesn’t.

I know the speech commands is listening correctly because I changed a button caption in code and a few seconds later the speech commands updated with its new value…without me posting any notification.

It seems to update the Front Window controls values independently of the Menubar.

Is it possible that passing the handle of the window or menubar is not working because I first must somehow convert the object into an NSObject and pass that or its ID instead?

This link uses the original string constant and has some Obj-C info:

https://github.com/ralcr/hxcocoa/blob/master/osx/appkit/NSAccessibility.h

[code]/*** Posting Notifications ***/

/* Posts a notification to accessibility client observers. Note that accessibility notifications are not NSNotifications and do not use the NSNotificationCenter mechanism. These notifications are received by client processes using the AX API defined in <HIServices/Accessibility.h> including AXUIElement.h.

For all notifications, the observer recieves the provided notification string and the AX API representation of the provided element.

For most notifications, the provided element is checked for observers of the provided notification.

For some notifications, the accessibility parent of the provided element is checked for observers instead. An example is NSAccessibilityCreatedNotification. It is impossible for a client to register to observe this notification on a new element, since the element does not exist yet. So, the function would be called passing in the new element, and the accessibility parent of the element is automatically checked for registered observers.

The following notifications check the accessibility parent of the provided element for observers: NSAccessibilityCreatedNotification, NSAccessibilityWindowCreatedNotification, NSAccessibilityDrawerCreatedNotification, NSAccessibilitySheetCreatedNotification, NSAccessibilityHelpTagCreatedNotification, NSAccessibilityRowExpandedNotification, NSAccessibilityRowCollapsedNotification.

For another set of notifications, the NSApp instance is always the observed element. An example is NSAccessibilityFocusedUIElementChangedNotification. The provided element would be the element that now has focus, and the function will automatically check NSApp for registered observers.

The following notifications always check for accessibility observers of NSApp: NSAccessibilityFocusedUIElementChangedNotification, NSAccessibilityFocusedWindowChangedNotification, NSAccessibilityMainWindowChangedNotification.

The rule of thumb is that the affected element should be passed into the function (the newly created element, the newly focused element, the row that was expanded, etc.), and the function will check for observer registrations on the correct element.

*/[/code]

See also: https://github.com/xamarin/xamarin-macios/blob/master/src/AppKit/NSAccessibility.cs

[code][DllImport (Constants.AppKitLibrary)]
static extern void NSAccessibilityPostNotification (IntPtr element, IntPtr notification);

	public static void PostNotification (NSObject element, NSString notification)
	{
		if (element == null)
			throw new ArgumentNullException ("element");

		if (notification == null)
			throw new ArgumentNullException ("notification");

		NSAccessibilityPostNotification (element.Handle, notification.Handle);
	}[/code]

Also shooting in the dark, tested it with no crashes, but I don’t know what I’m looking for if it were actually working…

[code]#if TargetMacOS and TargetCocoa then
declare function NSClassFromString lib “Cocoa” ( className as CFStringRef ) as Ptr

// — Collect the menubar from [NSApplication sharedApplication]
declare function sharedApplication lib “Cocoa” selector “sharedApplication” ( NSApplicationClass as Ptr ) as ptr
declare function mainMenu lib “Cocoa” selector “mainMenu” ( NSApplicationInstance as ptr ) as Ptr

declare sub NSAccessibilityPostNotification lib “Cocoa” ( instance as ptr, notification as CFStringRef )
const NSAccessibilityValueChangedNotification = “AXValueChanged”

NSAccessibilityPostNotification( sharedApplication( NSClassFromString( “NSApplication” ) ), _
NSAccessibilityValueChangedNotification )
#endif[/code]

Thanks, Sam. I modified your code slightly after looking at this thread (https://forum.xojo.com/37244-rename-app-on-open/0) …

[code]#if TargetMacOS
'NSMenu* theMenu = [[[NSApp mainMenu] itemAtIndex:0] submenu]; theMenu.title = @“Test”

Declare Function NSClassFromString Lib "Cocoa" (className As CFStringRef) As Ptr
Declare Function sharedApplication Lib "Cocoa" Selector "sharedApplication" (aNSApplication As Ptr) As Ptr
Declare Function mainMenu Lib "Cocoa" Selector "mainMenu" (aNSApp As Ptr) As Ptr
Declare Function itemAtIndex Lib "Cocoa" Selector "itemAtIndex:" (aNSMenu As Ptr, index As Integer) As Ptr
Declare Function submenu Lib "Cocoa" Selector "submenu" (aNSMenuItem As Ptr) As Ptr
Declare Sub setTitle Lib "Cocoa" Selector "setTitle:" (aNSMenu As Ptr, NewTitle As CFStringRef)


DIM NSApp As Ptr = sharedApplication(NSClassFromString("NSApplication"))
DIM NSAppMainMenu As Ptr = mainMenu(NSApp)
DIM applicationNSMenuItem As Ptr = itemAtIndex(NSAppMainMenu, 0)
DIM applicationNSSubMenu As Ptr = submenu(applicationNSMenuItem)
setTitle applicationNSSubMenu, "hello"

#endif[/code]
…and it compiles and renames the menubar top level item (so we know the object reference is correct) but unfortunately it still doesn’t register the menubar update. I tried replacing “Cocoa” with “AppKit” and “AppKit.framework” and “AXValueChanged” with “NSAccessibilityValueChangedNotification” but still no joy.

I found this link (https://github.com/cdelouya/champ/blob/master/sdk/MacOSX10.9.sdk/System/Library/Frameworks/AppKit.framework/Versions/C/Headers/NSAccessibility.h) and the line below:

APPKIT_EXTERN void NSAccessibilityPostNotification(id element, NSString *notification);

And am wondering whether the const string is the problem and that maybe it should be a pointer to a const instead?

You’re welcome, I can see that you’re learning this fast!

Technically “Cocoa” encompasses almost all the Apple Frameworks. I used to write declares that targetted the exact framework, only to find that with one system update, Apple changed the name of the Framework (or moved the library into a different framework) and all my declares were then shot. So my rule is use “Cocoa” unless the Framework isn’t covered by it.

I have a little hand made application that I use for asking the Framework the values of String constants and “AXValueChanged” is what it gives me.

Of course it’s possible to probe the Framework at runtime to get the actual instance of the const, but I’ve not had a problem doing this before with string constants (doesn’t mean that it can’t have a problem).

I am beginning to wonder if it’s not about getting the right API, I wonder if the OS actually supports this from a menu. There’s a couple of reasons as to why I’ve come to this conclusion.

  1. When using Xcode, menus are store in the nib/xib file with the resources and are ‘generally’ loaded from that file when the application opens.
  2. I don’t recall ever seeing an application where the menu changes drastically, just because I’ve never seen it, doesn’t mean it doesn’t exist. Just saying that it’s not commonplace.

How much of a challenge would it be to redesign so that’s only one menu?

Hi Sam. Thanks for your comments.

I think you may be right. I have opened up the Accessibility Inspector in Xcode and see that the dynamically added menu item shows (without me having to post a notification about it) but it just does not register the change to the available Speech Commands window.

It is possible to get rid of the alternate menubar and just dynamically add a top level “accessibility helper menu” item when needed and remove it when it is not but that still would not work as we have already discussed unless we can find a way to force Speech Commands and other Accessibility Client Observers to update and register the change.

It registers the change if you lose focus to the desktop or another app but that’s the only time I see it doing so.

The only other way I can think of is to bury the accessibility helper menu as a sub menu within a top level menu but again that’s not ideal.

Sorry I forgot to mention that on the Mac, you can have hidden menu items, so they’re all loaded on the menu, but you can use code to hide and show them as neccersary; I don’t know if it will help or not, but it might be worth a shot.