Extract a frame from a movie

Been looking through the documentation on using Movies now that I know editable movies are gone :frowning:

I see that the movie class has a ‘handle’ property, but the documentation doesn’t make this clear what the underlying object is, so I’m not sure what Apple API I should be looking for.

There’s a bunch of things I need to collect from a movie.

  1. FPS (if possible).
  2. Other meta data, such as aperture and shutter speed (again if possible).
  3. A full sized frame at a certain time in the movie, the preview shown to the user may or may not be scaled, but I must have the full sized frame.

Anyone have any clues, I see that the moviePlayer can be used with drawInto, so I might have to do some hacking.

I’ll take the frame as a bitmap image, either a Xojo Picture, CGImage or NSImage.

If you have access to Monkeybread plugins, you can use QTFrameExtractorMBS to pull frames from a movie object.

Thanks Jared, but I’m targeting the App Store where Quicktime is prohibited :frowning:

I might just dig into to the Apple docs on AVFoundation and see what I can find.

http://documentation.xojo.com/index.php/Control.Handle (this is what the MoviePlayer Handle property links to)…

Apparently this will allow you to create a thumbnail from a specific time stamp in the video file…
https://developer.apple.com/reference/avfoundation/avassetimagegenerator

And before Christian pipes up, this is available in MBS as well (so might save you some time doing declares and whatnot)
https://www.monkeybreadsoftware.net/class-avassetimagegeneratormbs.shtml

AVAssetTrackMBS has nominalFrameRate.
http://www.monkeybreadsoftware.net/class-avassettrackmbs.shtml

so for all video and muxed tracks you can query that.

I’ve been attempting to do some stuff with this today and I’m getting nowhere fast.

  1. The Movie Player.handle refers to a XOJMoviePlayer and I have no idea what that actually is, AVFoundation documentation has a lot of things that it could be.
  2. Movie.handle always returns 0 (<https://xojo.com/issue/48556>).
  3. [AVAssetImageGenerator copyCGImageAtTime:actualTime:error:] constantly returns an error; to which after hours of Googling, I can find no solution.
  1. Most of the examples for the above mentioned function, seem to refer to iOS, I can’t seem to find anyone who’s using it on the macOS.

Anyone done this, boy do I miss QuickTime where ■■■■ worked and it was easy to do.

Right, on top of that moviePlayer.drawInto doesn’t actually draw the frame of the movie, it draw a blank player instead. Seems like this project is dead before it started.

[h]HOLY BANANAS BATMAN![/h]
For [AVAssetImageGenerator copyCGImageAtTime:actualTime:error:] CMTime.flags must be 1 or the function doesn’t work.

So I’m leaving this here for the future and so that others who run into this problem can try this solution also.

Have you tried my plugins?
I think I have a Movieplayer function to get AVPlayer.

So after I donned my safari hat and went exploring; I can conclude the following.

The Xojo Movie Player is a regular or custom NSView, which uses a “AVPlayerLayer” (CALayer) to display the movie, from this layer you can get the “AVPlayerItem” and from that you can get the “AVAsset”.

I hope that this helps someone in the future.

[quote=340732:@Sam Rowlands]So after I donned my safari hat and went exploring; I can conclude the following.

The Xojo Movie Player is a regular or custom NSView, which uses a “AVPlayerLayer” (CALayer) to display the movie, from this layer you can get the “AVPlayerItem” and from that you can get the “AVAsset”.

I hope that this helps someone in the future.[/quote]

It will be nice of this information was given on the docs page

But it’s an implementation detail you can’t relay on.

Going into the future we simply don’t know; but for those who decide to extend the movie player or want to do movie editing it’s a good reference point.

Also I think the way how Xojo have done it is incredibly smart; it makes their application far more compatible and flexible than if they’d adopted the AVPlayerView. It’s actually quite inspiring and I’m going to see if I use the same trick with Metal.