Catalina Screen Recording Permissions

My app captures thumbnail images from a HTMLViewer by taking a screenshot. It needs to do this because it’s the only way I’ve found to fully capture everything in the window - some assets such as video, Flash/SWF, and WebGL playback don’t show up when using the OS APIs to caputre a HTMLViewer’s content.

This triggers the “App X needs to record the screen” dialog.

Problems:

  • there doesn’t seem to be a way to customize the string explaining why the app needs this permission. With other permissions, you can set a plist string such as
NSPhotoLibraryUsageDescription=MyApp can use images, movies, and metadata such as title and description when you drag & drop items from the Photos app.

but I don’t see anything equivalent for Screen Recording. Am I missing something?

  • Once you grant permission, the app must quit and restart. This is awkward for the end user.
  • Is there any way for the app to determine if the permission already exists? Or is the only way to try to take a screenshot and fail, triggering the OS dialog?

Do you tried Self.DrawInto ?

That doesn’t help - the underlying bug seems to be in WebKit itself: See 198107 – WKWebView takeSnapshotWithConfiguration does not take a snapshot of html5 video (youtube.com) and WebGL (i.e. hardware accelerated)

Apps need to be restarted: users get confused between the different security settings (automation, full hard disk access). But they understand just fine that they need to restart the app.

You try, if this fails then the security dialog is presented. After restart you try again.

The plist entry you want is NSCameraUsageDescription. I cheated and peaked into Screenium.

Some info that may be helpfull:
https://developer.apple.com/documentation/avfoundation/cameras_and_media_capture/requesting_authorization_for_media_capture_on_macos

Unfotunately, setting the NSCameraUsageDescription plist does not change the dialog for screen recording - it still has the default:

(Tested in Catalina dev beta 11).

Beatrix and Derk: it seems that Media Capture means “camera and microphone” and Screen Capture is something different.

Wow, this is really confusing. Here’s what I’m seeing in Catalina Dev beta 11:

  • the first time your app tries to capture the screen, it works.
  • Then, after the screencapture succeeds, you see the message ““MyApp” would like to record this computer’s screen.Grant access to this application in Security & Privacy preferences, located in System Preferences”
  • Even if you don’t give permission, your app can do screen recordings
  • After that first time, you won’t see the message again.

It’s almost like Apple has put in a “Allow screen capture while this app is running” feature which is different than “allow screen capture while this app is in the background”. This is kind of how iOS works. But none of this is documented.

Possibly relevant:

  • my app is only screencapturing from windows it owns
  • the actual capture is being done by a helper app that lives in the main app’s bundle and is signed with the same bundleID
  • I’m using ScreenshotRectMBS() to do the actual screen capture.

Not clear if this is a Catalina feature, security bug, or what…

If you google you will see applle mailed alot of developers, about screen capture and privacy reasons probably they have purposely left that the list key out ?

I havn’t seen a special plist key for screen record or capture.

Get used to it; this is the modern Apple. I also love when they create a page for the API, but don’t provide any information on what it or parameters do.

I recently had to choose between 3 different functions to solve a problem, the one that worked correctly on 10.14.6 was the one that had no documentation just the function name, the other two were 90% of what I wanted.

To be pedantic, your helper has a separate PID and doesn’t own those windows you’re capturing. Even if you correctly share the same security protocols, it still doesn’t own those windows. So it’s one process trying to capture another process’ windows.

Which API?

[quote=456519:@Sam Rowlands]Get used to it; this is the modern Apple. I also love when they create a page for the API, but don’t provide any information on what it or parameters do.
[/quote]

The people at Apple are great (when you can interact with them) but their public facing “API” (ha ha) is terrible, like you say.

Actually I was not clear: in this case the helper app creates its own window, loads the content, then screenshots itself. So it does own the window in this scenario.

Good question. It’s been a struggle over the years because of changing content (SWF/Flash and QuickTime which I’m no longer using, HTML5 video, and now WebGL) as well as changing software: Carbon->Cocoa->WebKit2->WebKit2 etc.

I was able to find a workaround in this case. The main problem I was having was that WebGL content would not show up when asking the HTMLViewer to render to an image. The solution was to do some WebGL trickery where I copied the canvas to another 2D canvas, which can be captured properly. Fortunately, I control the HTML/Javascript so this was easy.

For getting the HTML image, I’m using WKWebViewMBS control and the takeSnapshot method here: https://www.monkeybreadsoftware.net/htmlviewermac-wkwebviewcontrolmbs-method.shtml#15
but I think other solutions will work.

As a result, I no longer need to record the screen, and Catalina no longer forces the app to get screen recording permission.