A MacOS Console app to display video

I think I want to create a Console app (which I know is not intended to have any windows or UI) which creates a window to display incoming video as efficiently as possible. The Console app would be controlled by a Desktop app which has all of the UI.
Since the Window class is not available in a Console app project, I think I would need to create an NSWindow via declares. This is where i’m out of my skill set.

The application EyeTV (Geniatech, formerly from Elgato Systems) appears to have this architecture.

Any pointers on how to accomplish this would be appreciated.

You’re approaching this backwards. Figure out what function you want to achieve, then figure out what technologies you need.

What is the purpose of the functionality?

To display video data as efficiently (i.e. glitch-free) as a stand-alone process. A desktop app having a single thread is not responsive enough to do this and maintain a responsive UI. Why do you see this as backwards? Are you familiar with the app I’ve mentioned?

After watching EyeTV in Activity viewer, i see its background process is called “Elgato MPEG-2 decoder” and so it may not in fact display the video, but simply decode it.
The app I’m trying to create will receive video data in packets of video-frames/audio-frames. I have successfully created and transmitted data in this format (NewTek’s NDI: https://www.newtek.com/ndi/) but my desktop app (on an older MacBook Pro) can only achieve about 20 frames per second. To receive, the process needs to be able to receive and display 30 FPS or better. …that’s the goal.

Well this isn’t the intended way to do things. You might wanna create a Desktop Application for this, that accepts Command Line arguments From the App.Open event check the values of
System.CommandLine

then you have a console application that’s basicly able to stdout (should) and have a UI on commnand.

Set the default window to none and do manage the code from within App.Open.

[quote=419643:@Tod Nixon]After watching EyeTV in Activity viewer, i see its background process is called “Elgato MPEG-2 decoder” and so it may not in fact display the video, but simply decode it.
The app I’m trying to create will receive video data in packets of video-frames/audio-frames. I have successfully created and transmitted data in this format (NewTek’s NDI: https://www.newtek.com/ndi/) but my desktop app (on an older MacBook Pro) can only achieve about 20 frames per second. To receive, the process needs to be able to receive and display 30 FPS or better. …that’s the goal.[/quote]

While that may be possible to do in xojo, i’d be using plugins for this. Try the MonkeyBreadSoftware plugins for mac, see if they help.

I am using the MBS Picture plugin for the transmit function. And I’ll probably need it to properly format received frame data.

That would give me a separate process whose only job is display, but it would appear as a separate app and UI to the user. How could I hide that reality and present a single app icon and UI to the user?

[quote=419643:@Tod Nixon]After watching EyeTV in Activity viewer, i see its background process is called “Elgato MPEG-2 decoder” and so it may not in fact display the video, but simply decode it.
The app I’m trying to create will receive video data in packets of video-frames/audio-frames. I have successfully created and transmitted data in this format (NewTek’s NDI: https://www.newtek.com/ndi/ ) but my desktop app (on an older MacBook Pro) can only achieve about 20 frames per second. To receive, the process needs to be able to receive and display 30 FPS or better. …that’s the goal.[/quote]
I suspect what’s happening (because this happens when processing video on the macOS) is that the console application you’re seeing is actually the decoder, it simply converts the data into something that something else can use, then sends it to the GUI application, which only has to display it (which it can do at > 60 fps, as most screens on the Mac are not faster than 60 fps, anything above becomes irrelevant).

I see this behavior when working with Apple’s AVFoundation, where a second process magically appears called somethingorratherDecoder.

Thanks Sam. That must be it. My plan is to start with a simple working version of a receiver, and then see how performance might be improved by offloading the network socket and transcode functions to a helper app, using IPC between apps.

IPC sockets are slow; I would recommend using shared memory, except that I’ve still to figure out which shared memory functions are suitable for the App Store. I spent a few days experimenting and figuring out how to do things with shared memory, only to discover that due to poor documentation I’d used functions that are not allowed with Apple’s modern “security” policies. The functions I used (that are not allowed) are listed here. https://forum.xojo.com/51306-shared-memory-app-sandbox

I would recommend using a tool other than Xojo, which is what I was trying to get at with my first post. Using the right tool for the job is very important when the results are mission critical such as software for the entertainment industry.

Xojo is great, it offers developers a lot of advantage in terms of ease of development - but it has it’s costs. Xojo is a terrible media player, and the controls it offers us for the job are severely limited.

@Sam Rowlands --I was wondering about that. I use IPC sockets often. That is very interesting information! And I’m doing in-house type projects that won’t ever need to pass the App Store.

@Tim Parnell --I don’t dispute there are more appropriate tools for this kind of project. But for better or worse I’ve been working with RealBasic-Xojo for over 20 years now. My case may look like a “hobbyist” to some, but Xojo has been a big part of my career, despite the fact i’m not a full time programmer and have a different job title. I learned Fortran & Assembly back in the mainframe days, and have always supported the dream of a full-featured rapid dev environment.

Stupid question: what do the shared memory functions from the MBS plugin use under the hood?

Could you direct that question to Christian Schmitz in a new thread? This sounds like a topic many of us will be interested in.

It would be nice to have an NDI plugin tbh :slight_smile:

+1 on the NDI plugin

I’m also looking at bring video from a video playout server which supports NDI, I’m currently pushing it out over UDP as H.264 which is processor intensive vs NDI and then using the MBS VLC plugin to display it in my project.