Symbol extraction from video

I use a Mac, but as I said earlier, this is just a pet project and I can’t justify the cost of a plugin license. Thanks anyway.

You could still apply for a trial license.

Sam Rowlands above told you it was possible without plugin. You will have to learn how to do it with declare.

To Christian S: Thanks for the suggestion, but a trial license, even if I got it to work for my needs, would only be an interim thing, and I would still have to eventually pay for the license.

To Michel B: Telling me it’s possible gives me some encouragement, but it doesn’t help me to do it. I started this thread on the assumption that it’s possible.

I’m still hoping that someone will point me to an example of Xojo (RealStudio) code to extract one frame at a time from a mpeg file, and then how to look at the pixels in that frame, without any kind of plugin.

Thanks, Tim, but if the project can’t justify the cost of a plugin license, how could it justify the cost of a consultant?

Well, if the project has no value, why do you do it at all?

And somewhere you got the Xojo license already, so you paid something, right?

Of course if you stay in Xojo without license and only use the app in the IDE, you can also just use the plugin in the IDE without a license. That’s fine.

As well as a trial license, which lets you build a prototype, so if you find someone who funds the project, you could than buy Xojo and plugin licenses as needed and add a consultant to help you.

[quote=379072:@Dan Paymar]To Michel B: Telling me it’s possible gives me some encouragement, but it doesn’t help me to do it. I started this thread on the assumption that it’s possible.

I’m still hoping that someone will point me to an example of Xojo (RealStudio) code to extract one frame at a time from a mpeg file, and then how to look at the pixels in that frame, without any kind of plugin.[/quote]

Sorry, Dan, I never needed it, so I never learned how to do it. And as professional, I do use MBS plugins instead of reinventing the wheel.

Now, after stating several times it was a pet project and you could not afford commercial shortcuts, you must accept to be a lot more modest, and patiently learn everything you need done. It is not as if nobody tried to put you in the right direction in this thread…

You can start by trying to port the image recognition algorithm, using photos instead of captured frames. The principle will remain the same.

You will not find here people who provide ready to use examples of things that they never did. I don’t recall ever seeing someone with a project quite like what you want to do. It is one thing to do ten lines of easy code to help a beginner, and another to write a difficult and complex program just to be nice to someone who seems both impatient, and not very willing to make a lot of personal efforts.

I doubt extremely much the code you want already exists. If you want your project to happen, you will have to learn by doing, like most of us did.

That’s the answer I give when someone asks specifically for free code, but hasn’t shown any interest in learning the basics for how something is built or why it works. We’re tinkerers and builders, we can help you learn how to do things but it takes a long time.

If you need something done now or want code examples without showing how you’ve tried, these aren’t free.

I’ve been programming computers since 1957, and in RealBASIC/RealStudio/Xojo for over 20 years, so I do have a little experience, and I’ve helped a lot of other people without asking for pay.

Xojo includes a lot of example code, but I couldn’t find anything on examining individual frames in a video file. Sure, I can load it into a memory block, but then what?

I guess what I need first is to know the actual format of a video file. If some format is easier to handle than mpeg, I know how to use a video converter, but where do I get info on the file’s internal format?

https://en.wikipedia.org/wiki/Moving_Picture_Experts_Group

https://ffmpeg.org/

I understand that MPEG is a compressed audio/video format. Perhaps my first step would be to expand it, drop the audio, and save only about every 5th frame in 720p size. I could also reduce the colors to monochrome to save space. But I can’t even start that without knowing the actual MPEG file format, and my searches have not yielded anything useful.

It may not be easy for video as there are many different types, and storage of video is looks like it’s quite complex. I started here Video file format - Wikipedia and it says the video file consists of a container format and a video coding format. Determining all of this is likely no easy task.

Mac OS has a system-level video framework called AVFoundation which handles opening of videos for you, but as has been mentioned in this thread you’ll need to use declares or plugins to use that. Figuring out whether you want to re-write that framework in Xojo or learn the declares or employ plugins is up to you. Each of the options has advantages and drawbacks, the first two options having the drawback of learning curve time.

You might also explore FFMPEG as Michel linked, but the open source library isn’t easy to use in conjunction with Xojo. Someone here (I forget who) is the author of iffmpeg/ffworks, and I believe that tool is mostly command line wrappers for the FFMPEG functions. That person if they stumble across this thread would be the most knowledgable on integrating the library with Xojo.

FFMPEG is also open source, which means you can use your 50 years of experience to peruse the code and understand how reading MPEG format is done. Again, opening you to the option of implementing the whole thing in Xojo if you so choose.

Thanks, TIm, for a long and thoughtful reply.

It looks like it may be easier to convert the file to GIF (“inefficient compression” and no sound) before trying to process it.

Much appreciate that, Dave.

In the mean time, I finally found a detailed description of the GIF file format at
https://www.w3.org/Graphics/GIF/spec-gif89a.txt
At least that gives me a starting point.

I am but a novice at this, but I would take Christian’s offer at face value. He is saying you can try his stuff free as a trial. You can even build the application (cause his plugins don’t prevent you from doing that) to try it further. And if you ask him nicely he may even give you some tailored examples with this plugins. To me that is worth something. I would at least try it to see what is possible, and then if you wish you can try to do it all in Xojo code once you prove to yourself it is doable and you like how it behaves.

As for what I’ve seen done/asked: search the forum for “frame grab”, “VLC”, “ffmpeg”, “face detection”. I think these things will get point you to what is out there. I recommend using the “better” search tool too: https://forum.xojo.com/30945-searching-tips

Sure, let me know when and I’d be more than happy to show you around the peninsular; just not right now, it’s freezing at only 22?c!

I can do that, but it will take me some time to extract the code from a project (I have already written and am using that code) and wrap it up so that others can use it. Just right now; I am really busy with a bunch of things (one of them being 64-Bit migration of existing applications).

There’s a bunch of ways to extract frames from video, probably the easiest is via CGImage, which can then be drawn into a Xojo picture for pixel analysis (or drawn into a CGBitmapContext created from a memoryblock and use the memoryblock to read pixel values).

You don’t need to understand any of the movie formats, however the second method for extracting frames is really interesting as it can return a ARGB based memoryblock or YUV in a variety of different settings. Understandably the quickest to read is in the video’s native format (which takes some deciphering to figure out), as modern processors include optimized paths for reading video (hence why you can watch a movie for 10 hours on a 2016 Macbook Pro, but only use Xojo for 3~5).

AVFoundation and CoreImage go hand in hand, Core Image is optimized to process frames in YUV or ARGB, so for image manipulation on a frame, I would would recommend Core Image. However if you want to use Core Image to edit Photos, I no longer recommend it unless you’re doing some filter chains or using 12 megapixels from Apple’s Professional camera (iPhone).

Christian Schmitz, sorry I missed your post. Apparently it came up in my system at the same time as two other responses, and I didn’t look back. I’ll respond now. Your text in quotes.

“Well, if the project has no value, why do you do it at all?”
I never said it has no value. What I said was it has little or no MARKET value.
It has value to me as a curiosity. I’m long retired and have time to satisfy my curiosity.

“And somewhere you got the Xojo license already, so you paid something, right?”
Of course, but I didn’t get it just for this. I’ve been using Xojo and its predecessors for over 20 years to develop Optimum Video Poker. You can see details on it at www.optimumplay.com.

“Of course if you stay in Xojo without license and only use the app in the IDE, you can also just use the plugin in the IDE without a license. That’s fine.
As well as a trial license, which lets you build a prototype, so if you find someone who funds the project, you could than buy Xojo and plugin licenses as needed and add a consultant to help you.”
I may take you up on the trial license. Right now I’m just investigating options.
Thanks,
Dan

Sam, thank you very much for your very generous offer. There is no rush for me, and I’m grateful for any help I can get.

But you say 22°C is freezing? I would consider that very comfortable, or even on the warm side if it’s humid, too. If that’s freezing, I wouldn’t want to be there when you consider it hot.