When loading a ProRes or similar high quality file into a movie player, is the data that’s displayed using the file’s bit depth? For example: Uncompressed 10 bit or ProRes 422HQ are both 10 bit file formats. ProRes 4444 is up to 12 bits.
When loading the file into a MoviePlayer in Xojo is the bit depth altered to something lower, or is the data there available at its native depth?
I want to extract a frame from the MoviePlayer object, but only if it’s the full image as it exists in the Quicktime file itself.
I’m sure Xojo is using the OS’s framework for displaying video, so your app will display whatever it deigns to give you. I would wager that it might in some cases scale the output to the display context - no need to spit 12 bits/channel to an 8 bit/channel display.
I’m just digging through the documentation right now, but the gist of it was that I’d use MoviePlayer.DrawInto to drop the frame as a picture into a canvas. It’s there that I’d be doing some manipulation of the image.
I’d need to be able to play, stop, frame forward and frame backward on the movie, and have the current frame show in the canvas. Realtime playback isn’t important here, but it does need to be able to display all of the frames without skipping, even if it can’t do it at its nominal frame rate.
This setup also needs to work with image sequences (a folder full of sequentially numbered TIFF images), so ultimately, I need the frame in a canvas to do what I need to do with it.
I can work with lower bit depth images in the canvas itself, and then process in the background at high bit depth using Xojo OpenCV-C which is where the heavy lifting in this app is done anyway - but if the image data itself is the native bit depth within Xojo then I’d just work with that. I suspect it’s not, but google searches are bringing me a lot of old forum posts saying so, and I don’t know if that’s changed in recent years.
Files like Uncompressed Quicktime or AVI basically are what they are and should be easy to open directly in OpenCV. Same with image sequences.
But formats like ProRes don’t have great OpenCV support natively, so I’d prefer to have that handled by the OS-level support. Presumably that’s also going to be faster because of the built in media encoder on most modern macs.
So in an ideal world, I get the 10 or 12 bit frame into Xojo so that once it’s there I can just work with it. But the sources will vary. MoviePlayer, in my past goofing around with it, was able to easily play very high resolution 4k+ ProRes files in a Xojo app in real time, which is why I’d like to use it to source the frames.
I’m not 100% sure this is going to be mac based. I may end up doing it on Windows. As long as Quicktime for Windows is installed, the ProRes files we would need to support should work there as well, I’d imagine. (haven’t tested that, though)
I should also say that in terms of capabilities, I don’t need anything other than play, stop, frame fwd/rev so nothing particularly complicated.
Basically all work that’s done in this app is done on a frame by frame basis, so all I want is to get the current frame into memory at its native bit depth so I can do my thing with it.
Quicktime for Windows has been discontinued for nearly a decade and you should not rely on it. You might want to explore the VLC capabilities of MBS or maybe try to extract frames by shelling to ffmpeg.
It’s already installed on the machines in question. We have used and will continue to use it as long as it works.
There is nothing wrong with the underlying Quicktime for Windows engine, the issues with Quicktime’s potential vulnerabilities were related to movies that had embedded links. In this context that isn’t even possible as none of the movies we’d ever load into this application would have that - they’re all created in-house with tools that can’t even do that. I haven’t even seen a Quicktime movie with embedded hyperlinks since the early 2000s, to be honest.
Yes, Apple has discontinued support for it on Windows, but it still works and we use it all the time.
If you can suggest an alternative for Windows that can reliably decode ProRes files as well as other high bit depth formats like Uncompressed, and that I can use from within Xojo, I’m all ears.
We don’t use nor do we recommend using VLC and honestly I wouldn’t even consider it, in terms of image quality. I’ve been doing work with digital video on the desktop for almost 30 years and VLC has never been a reliable tool for ProRes - it has always been buggy. It’s fine for consumer-level video and compressed formats like MP4, but we’ve seen lots of issues that come and go with it over the years related to how it decodes ProRes, especially if that file happens to have high bit depth audio embedded.
Same goes for ffmpeg, which is a reverse-engineered ProRes encoder/decoder and doesn’t produce images of the same quality as Apple’s encoder or those made by Apple-blessed third party vendors on Windows. We’ve tested this extensively.
I understand not relying on Quicktime for Windows in a commercial app. But we’re talking about something for internal use in a controlled environment.
At the moment, though, this is all theoretical until I have some time to try it out, I’m just looking at the documentation to see what’s built-in to Xojo that I can use with minimal friction.
My decision about the platform is largely going to be based on which is easier to do. I’m leaning towards Windows at the moment. All of this comes natively on the Mac, yes, but most of our work is done on Windows machines, and those machines are capable of connecting to our 40Gbps network. None of the macs have PCIe slots (except for old, slower MacPros that we’re working to decommission), so we can’t run NICs at that speed on a mac and that makes a big difference when you’re working with a 6k-8K 16bit TIF file and you’re simultaneously reading/writing over the SAN. 10GbE doesn’t cut it. Yes, on paper it should, but we’re not dealing in theoretical numbers, we’re basing this on real-world experience actually using these kinds of files on our system, and we will probably need this to be on the 40G network, which is why I’m leaning towards Windows at the moment.
Trust me, we’ve been down this road. I am using a thunderbolt expansion chassis on my MacStudio and have tried both Chelsio and Mellanox 40GbE NICs (the only ones I’m aware of that work with Mac OS), and neither is able to achieve the same speeds as the same cards on Windows. Mellanox only has built-in drivers for mac, which are basically not tunable, and Chelsio’s are pretty much in a beta state for newer OS versions.
The network we have is 40GbE, not the more common 25GbE, for which there is good support. Our switch can’t do 25GBE, so making that happen is a massive expense and disruption to our workflow, so it’s not really an option. We’ve had this setup for almost 10 years now. It works really well with Windows and our older Macs with PCIe cards (though their speeds are bus-limited being as old as they are). But Apple being Apple, well they know best. I’ve always found it funny that Jobs and Ive were so obsessed with decluttering yet they made decisions that lead to a completely cluttered desktop for anyone wanting to expand the machine’s capabilities.
Quicktime isn’t supported on Windows 10 or later – if you’ve gotten it to work, congrats, now don’t change anything about the machine. Of course, the risk is that some update to, or newer version of, Windows will cause it to stop working, and you’ll be stuck keeping the system running on old, potentially insecure software.
Poking around, I see that Apple does provide native support for accessing ProRes in the various Adobe video applications. If you do decide to focus on Windows, I wonder if there’s a way to hook into those plugins.
Otherwise – I’d say focus on the Mac and solve the other problems (network bandwidth, etc). ProRes support on the Mac is always going to be superior to any other platform, so if image quality if your ultimate priority, it seems a simple choice.
Are you willing to look at new Mac Pro towers, with full PCI slots? I see you discussed this issue in another forum and were looking at getting some cards from China to experiment with.
There are more factors than I’ve mentioned. I mean, I can really get into if you want, but we have been dealing with this for some time and are actively moving away from Apple at this point because of the unreasonable cost of new Macs (once you factor in all the extras you need to do what we need to do), the general inflexibility of Apple, the difficulty of running our SAN on newer OS versions (they make it nigh on impossible for the SAN vendor to do their thing anymore, because what they need to do happens at a pretty low level). it’s an incredibly reliable, high performance system we’ve been using for years but in recent Mac OS iterations it’s taken them longer and longer to release drivers because Apple keeps tightening the screws and making it harder for them, and in turn, for us.
I’ve just had this conversation so many times with people I find it kind of exhausting to have to explain it. I appreciate the suggestions, but we’re at a stage where we are only using Macs to capture videotapes (and not for much longer), and for my desktop machine since I prefer it from a user perspective. But everything else is Windows.
BTW, Adobe’s ProRes is proprietary as far as I know, only usable within their ecosystem. It was only recently that Apple allowed Blackmagic Design to write ProRes files on Windows (like a few months ago), after many years of keeping it off limits - likely because they thought Resolve was a threat to Final Cut Pro (which nobody uses seriously anymore since Apple turned it into a high end iMovie with FCP X).
That said, ProRes is only one of the formats we need to handle. I dunno. Maybe we build two versions - one on Windows that deals with uncompressed image sequences and one on Mac that does ProRes. That seems less than ideal.
Maybe you make a Mac version that acts as an accessory to the Windows version. Put a Thunderbolt card in your Windows machine and it can (theoretically) do IP over Thunderbolt at 40Gb to the Mac. Your Windows app could throw the video file over the connection to the Mac app and then control it remotely, requesting frames. This could be a fairly inexpensive Mac Studio, as they all come with Thunderbolt 4.
This could be a way for you to structure your app for all manner of weird corner cases. For example, if you ever got into a spot where you had to maintain a machine on Windows 10 for access to some codec, this would be a secure way to do it: the Windows 10 machine could be configured in such a way that it has no access to the Internet at large, only communicating with a controlling machine through a dedicated IP over Thunderbolt connection. The main machine could then be merrily upgraded to Windows 3000 for general best practices and security.
Just for the sake of completeness, the MBS plugins provide a way to extract each frame of a movie, that you can directly get as a picture object (without using DrawInto). This is Mac only, though.
Thanks. I’d prefer to avoid plugins, but this might be a way to go if I go mac. It doesn’t seem that there’s a way to use MoviePlayer.DrawInto to get the current frame into a picture. When I tested it last night at home, I was just getting a blue screen, despite the frame in the movie player having changed. I was calling DrawInto from the Movie Players PlaybackStopped event, and it was definitely doing …something. Just not putting the image in the picture
Of course, the risk is that some update to, or newer version of, Windows will cause it to stop working, and you’ll be stuck keeping the system running on old, potentially insecure software.