Move mouse cursor and click by code for Mac

I have completed a method for Windows ( https://forum.xojo.com/12259-move-mouse-position-click-by-code )

Now I want to do the same for Mac OS X. The old AppleScript command click at {x,y} has been brutally retired with Mavericks, so it cannot be used.

I scoured the Internet and found Mousetools which is meant to be used from the command line. I extracted the useful methods from the source and am trying to slowly find my way to Xojo declares.

I found the Apple doc page for what is used : https://developer.apple.com/library/mac/documentation/Carbon/Reference/QuartzEventServicesRef/Reference/reference.html

Unfortunately, I find that documentation a lot harder to adapt than MS. To start with, I do not know what the library is, so I do not see how to build the declare. I searched for the function names in this forum unsuccessfully, and all I found was in http://forums.realsoftware.com/viewtopic.php?f=7&t=25915&hilit=CGEventTapOptions but no solution was ever found there. Interestingly, it all started from the impossibility to give focus to the HTMLViewer, and like the OP there, I want to click in it so it gets the focus.

My inspiration about Mac declares comes from the brilliant teachings of Sam Rowlands in https://forum.xojo.com/5179-mac-os-custom-fonts where I found at least this :

// - As we had a valid CFURLRef from a 'Create' or 'Copy' API, we must release it. declare sub CFRelease lib "Cocoa" ( ref as ptr ) CFRelease CFUrlRef

So I believe this is the way to CFRelease(moveMouse); and the like.

Now, for other functions.

Apple says for one of the functions :

[code]CGEventCreateMouseEvent
Returns a new Quartz mouse event.

CGEventRef CGEventCreateMouseEvent (
CGEventSourceRef source,
CGEventType mouseType,
CGPoint mouseCursorPosition,
CGMouseButton mouseButton
);
Parameters
source
An event source taken from another event, or NULL.
mouseType
A mouse event type. Pass one of the constants listed in “Event Types.”
mouseCursorPosition
The position of the mouse cursor in global coordinates.
mouseButton
The button that’s changing state. Pass one of the constants listed in “Mouse Buttons.” This parameter is ignored unless the mouseType parameter is kCGEventOtherMouseDown, kCGEventOtherMouseDragged, or kCGEventOtherMouseUp.
Return Value
A new mouse event, or NULL if the event could not be created. When you no longer need the event, you should release it using the function CFRelease.[/code]

Here below CGEventRef moveMouse = CGEventCreateMouseEvent(sourceRef, kCGEventMouseMoved, CGPointMake(x, y), 0);

SourceRef is in a type that does not seem to exist in Xojo. Is it possible to synthesize a Quartz event source ?

declare function CGEventCreateMouseEvent lib "Cocoa" ( sourceRef as ?, mouseType as MouseEvent, mouseCursorPosition as RealBasic.Point, MouseButton as integer )

I am stuck there.

Any help will be greatly appreciated.

Of course, I can still call Mousetools from the command line, but it would be great to be able to do a declare.

[code]// global variables
CGEventTapLocation tapLocation;
CGEventSourceRef sourceRef;
// Variables
sourceRef = CGEventSourceCreate(kCGEventSourceStatePrivate); // kCGEventSourceStatePrivate is a constant ? Value ?
kCGEventMouseMoved // constant NX_MOUSEMOVED, not found value ?
CGPointMake(x, y) // = new realbasic.point(x,y)
tapLocation = kCGHIDEventTap; // used when specifying the tap location for CGEventPost
moveMouse // constant ? Value ?
mouseEvent // constant ? Found this : CGEventRef mouseEvent = CGEventCreate(NULL);

void moveMouseToPoint(float x, float y) {
//CGWarpMouseCursorPosition(CGPointMake(x, y));
CGEventRef moveMouse = CGEventCreateMouseEvent(sourceRef, kCGEventMouseMoved, CGPointMake(x, y), 0);
CGEventPost(tapLocation, moveMouse);
CFRelease(moveMouse);
} // mouse jumps directly from its current position to the new position and becomes visible, origin of CGPoint must be top-left

void performLeftClick(CGEventFlags modKeys) {
// get the current mouse location
CGEventRef mouseEvent = CGEventCreate(NULL);
CGPoint mouseLoc = CGEventGetLocation(mouseEvent);
CFRelease(mouseEvent);

// click mouse
CGEventRef clickMouse = CGEventCreateMouseEvent(sourceRef, kCGEventLeftMouseDown, mouseLoc, 0);
if (!modKeys == 0) CGEventSetFlags(clickMouse, modKeys);
CGEventPost(tapLocation, clickMouse);
CFRelease(clickMouse);

// release mouse
CGEventRef releaseMouse = CGEventCreateMouseEvent(sourceRef, kCGEventLeftMouseUp, mouseLoc, 0);
CGEventPost(tapLocation, releaseMouse);
CFRelease(releaseMouse);

} // left-click at current mouse location, we pass the modifier keys mask in case we want to shift-click[/code]

I needed to move the mouse a while back, so here is my code. This code will center the mouse in the middle of a window.

[code] //Structure CGPoint:
//////////
//x as single
//y as single
//////////

Declare function CGWarpMouseCursorPosition lib “ApplicationServices” (newPoint as CGPoint) as Ptr
dim pt as CGPoint
//center mouse in the window
pt.x = self.Width/2 + self.Left
pt.y = self.Height/2 + self.Top
call CGWarpMouseCursorPosition(pt)[/code]

I’ll look into clicking the mouse now.

[quote=91562:@Jason King]I needed to move the mouse a while back, so here is my code. This code will center the mouse in the middle of a window.

[code] //Structure CGPoint:
//////////
//x as single
//y as single
//////////

Declare function CGWarpMouseCursorPosition lib “ApplicationServices” (newPoint as CGPoint) as Ptr
dim pt as CGPoint
//center mouse in the window
pt.x = self.Width/2 + self.Left
pt.y = self.Height/2 + self.Top
call CGWarpMouseCursorPosition(pt)[/code]

I’ll look into clicking the mouse now.[/quote]

Thank you Jason. This is much simpler than what I had found !

This frankenstein code will perform a left click and trigger the mouse down and mouse up (if mouse down is returned true) events of a control or button. I have noticed however that the function must be called twice in order to trigger the action event of a button, even though the mouse down event is still called if the function is called just one time. Maybe someone else will be able to explain a reason for this?

[code] //declare necessary functions, etc
Declare sub CFRelease lib “Cocoa” ( ref as ptr )

'CGEventRef CGEventCreate (
'CGEventSourceRef source
');
Declare Function CGEventCreate lib “ApplicationServices” (CGEventSourceRef as ptr) as Ptr

'CGPoint CGEventGetLocation (
'CGEventRef event
');
Declare Function CGEventGetLocation lib “ApplicationServices” (CGEventRef as Ptr) as CGPoint

'CGEventRef CGEventCreateMouseEvent (
'CGEventSourceRef source,
'CGEventType mouseType,
'CGPoint mouseCursorPosition,
'CGMouseButton mouseButton
');
'/* mouse events /

'#define NX_LMOUSEDOWN 1 /
left mouse-down event /
'#define NX_LMOUSEUP 2 /
left mouse-up event /
'#define NX_RMOUSEDOWN 3 /
right mouse-down event /
'#define NX_RMOUSEUP 4 /
right mouse-up event /
'#define NX_MOUSEMOVED 5 /
mouse-moved event /
'#define NX_LMOUSEDRAGGED 6 /
left mouse-dragged event /
'#define NX_RMOUSEDRAGGED 7 /
right mouse-dragged event /
'#define NX_MOUSEENTERED 8 /
mouse-entered event /
'#define NX_MOUSEEXITED 9 /
mouse-exited event */
Declare Function CGEventCreateMouseEvent lib “ApplicationServices” (CGEventSourceRef as ptr, _
CGEventType as UInt32, position as CGPoint, CGMouseButton as UInt32) as ptr
const kCGEventLeftMouseDown = 1
const kCGEventLeftMouseUp = 2

'void CGEventPost (
'CGEventTapLocation tap,
'CGEventRef event
');
Declare sub CGEventPost lib “ApplicationServices” (CGEventTapLocation as UInt32, CGEventRef as ptr)
const tapLocation = 0

'CGEventSourceRef CGEventSourceCreate (
'CGEventSourceStateID sourceState
');
Declare Function CGEventSourceCreate lib “ApplicationServices” (CGEventSourceStateID as UInt32) as ptr
const kCGEventSourceStatePrivate = -1

//now for the clicking
//get mouse location
dim CGMouseEvent as Ptr = CGEventCreate(nil)
dim mouseLoc as CGPoint = CGEventGetLocation(CGMouseEvent)
CFRelease(CGMouseEvent)

//create event source
dim sourceRef as ptr = CGEventSourceCreate(kCGEventSourceStatePrivate)

//press down
dim clickMouse as ptr = CGEventCreateMouseEvent(sourceRef, kCGEventLeftMouseDown, mouseLoc, 0)
CGEventPost(tapLocation, clickMouse)
CFRelease(clickMouse)

//release mouse
dim releaseMouse as ptr = CGEventCreateMouseEvent(sourceRef, kCGEventLeftMouseUp, mouseLoc, 0)
CGEventPost(tapLocation, clickMouse)
CFRelease(releaseMouse)[/code]

Note - if you check the console you will see the error “kCGErrorIllegalArgument: CGEventPost: invalid event” if you do not return true in the mouse down event of whatever you are pressing since the function attempts to simulate both the press down and release of the mouse.

Here is an example project with the function and the previous function to move the mouse position:
https://www.dropbox.com/s/jajxbkhj0rdtldv/mouse%20manipulation.xojo_binary_project

Jason, you rock !

I was still painfully assembling my declares and wondering how I could find the value of kCGEventLeftMouseDown…

This works but you are right, it takes sending two clicks to have the button action event fire.

I have tried to set the click to hit the terminal on my dock : first click does a right click, second click actually opens the terminal. It does the same for all items I tried in the dock. It clicks the menu edit fine, though.

Finally, I tried the click on an HTMLViewer, to click a link. The same as button happens. First time nothing. Second time the pointer changes for the finger pointer, and the HTMLViewer displays the link.

There must be some mystery at work…

In the meantime, maybe the workaround is simply to call the method twice ? I know, not very clean uh ?

Thank you for helping me through this. It would have taken me weeks :slight_smile:

[quote=91595:@Michel Bujardet]There must be some mystery at work…

In the meantime, maybe the workaround is simply to call the method twice ? I know, not very clean uh ?[/quote]

I agree that something funny must be going on in the background and the workaround would seem to be to call the method twice, although like you I don’t particularly like it. Hopefully someone will know more about what’s going on behind the scenes and can explain that.

You’re welcome, I’m glad I could help :slight_smile:

[quote=91578:@Jason King]Here is an example project with the function and the previous function to move the mouse position:
https://www.dropbox.com/s/jajxbkhj0rdtldv/mouse%20manipulation.xojo_binary_project [/quote]

Thanks, very helpful.

an additional method to place mouse in center of a Control

Sub SetCursorToRectControl(r as RectControl, w as Window)
  //Structure CGPoint:
  //////////
  //x as single
  //y as single
  //////////
  
  Declare function CGWarpMouseCursorPosition lib "ApplicationServices" (newPoint as CGPoint) as Ptr
  dim pt as CGPoint
  //center mouse in the window
  pt.x = w.left + r.Left + r.Width/2
  pt.y = w.top + r.top + r.height/2
  call CGWarpMouseCursorPosition(pt)
End Sub

SetCursorToRectControl (btn1, mainwin)

First time sets the focus, second time causes the Action?

I stumbled across this thread whilst searching for some declares to send a keyboard event to another application. Big thanks to Jason and Michael for doing all the hard work, and I managed to cobble together the necessary code to achieve my goal.
I’ll post the keyboard event declares I created when I’m next at my computer should anyone have a need for this functionality.

Regarding the problem of having to call the method twice, I think there might be an error in the 2nd from last line of code. Shouldn’t it read:
CGEventPost( tapLocation, releaseMouse)
Instead of
CGEventPost( tapLocation, clickMouse)

Thanks once again

Thank you, Axel S. This is great stuff. It’s nice to have the code demonstrated using an example project.

Note that this does not work in a sandboxed environment, and will not be accepted in the Mac App Store.

Thanks for all the replies

Axel version works great for Mac
Michel version https://forum.xojo.com/12259-move-mouse-position-click-by-code-windows/0 works great for windows.

Lennox