I have no immediate need for this, but I am using OpenCV with non-xojo apps on the Raspberry Pi. I would definitely like to leverage the work to be able to use it on windows and mac and linux without requiring separate installs of everything. I’ve got it working on the Mac via the command line tools but the install and build process were definitely not straight forward. If you did decide to dive into this, especially for the machine vision stuff, I would definitely find a use for it in the future 
I think it goes without saying that we’d also be interested in this. OpenCV is a massive project, and it’s a lot of work to get it talking to Xojo. I wonder, @Christian_Schmitz , would it be more appealing if we came up with a subset of functionality that would be needed to get something like this going, and then you could add more on an as-needed basis?
If so, I’d be happy to come up with a list of stuff we need in the short term. It’s definitely only some of what OpenCV does, though obviously others would have other needs and that may expand the list to the point where it’s too large again!
In the mean time I’d settle for a library I can declare into, but such a thing doesn’t seem to exist for 64bit and newer versions of OpenCV.
If we had more than one or two of us who were interested we might be able to pitch in a little to make it worth Christians while 
I know I’d be willing to but cannot fund the entire project myself until I have a direct need. I know I’ll be reaching that point eventually but right now there is no money to go around there so I’d be paying it forward. Which would make me perfectly happy to do, just not the entire cost. I wonder if we could start a thread to gather others to throw in a little cash to make it happen…
We’re in the same boat - 2 years behind on development of this film scanner we’re making, and budgets stretched thin. But we could contribute some as well.
Here is the state of things with OpenCV-Xojo, as I understand it:
- The OpenCV-Xojo project relies on OpenCVX, which provided a dynamic library tied to OpenCV that could be used in Xojo
- OpenCV-Xojo calls a lot obsolete OpenCV functions, so a lot would need to be updated
- The OpenCVX project relied on the OpenCV C-API which is now largely deprecated. Even the OpenCV Doxygen pages are blank for the C-API for most modules and its use is actively discouraged in places like stackexchange.
I’ve been talking with the person who wrote OpenCV-Xojo, and he did some tests to see if he could get it running but it’s too much work at this point. And with the reliance on OpenCVX, which hasn’t been updated in years, I’m not sure it even makes sense anymore.
So my questions are:
- Can Xojo declare into a C++ library or does it have to be plain C?
- If not, would it make sense to create a DLL/Dylib/SO that exposes some of the OpenCV C++ functions as plain C?
On the latter question, we would be willing to pay someone to do that (depending on the cost), for at least the subset of OpenCV functionality we need. (most likely core, imageproc, objdetect, photo and stitching. Maybe one or two more). We’re not interested in the video stuff.
I’m not sure what the cost would be on this but we have a few people we’ve worked with on this kind of stuff in the past and I can find out. If we were to get a few people together we could possibly add more functionality. I’m not opposed to making that code public, though maintenance might become an issue.
You need a C/C++ library exporting plain C functions.
DLL/dylib/so, so you can use it with declares.
I have seen a few requests for OpenCV, but this is a huge library.
Someone may need to define what they really need, pick a few functions and then we can look into interfacing it.
That’s what I figured. Thanks for confirming.
As I mentioned above, the functionality I’d be looking for would be in core, imageproc, imgcodecs, objdetect, photo and stitching, from the Main Modules (see: OpenCV: OpenCV modules ), for OpenCV 4.5.1 (the latest stable release).
For us, the video stuff isn’t really relevant at this time, though I’m sure it would be for others. I’m just diving into OpenCV in python right now, to develop some performance test setups and basic algorithms for the work we need to do, so I don’t know exactly which functions we’d need right now, but I think it’d be a subset of the ones listed above, to start.
how about creating a c++ commandline tool and start this from xojo?
This might work for small images, but as I mentioned above, we’re working with 300MB images (16 bit 14192x10640 pixels at a rate of about 3-4 frames per second). Unless we can somehow pipe the image from Xojo to the command line app, and then back into Xojo without hitting the disk, there’s too much overhead.
You could use our shared memory class in Xojo, pass identifier to helper app and have it read the shared memory block.
Does that work with a helper app that’s not written in Xojo? How would you pass the image data (already in memory, not a file) from Xojo to a command line application (say, a Python script), and then get the result back into Xojo?
If you like to go that way you need to check what functions python offers for shared memory.
I may help on details how the plugin does it.
So it looks like newer versions of python can do this. multiprocessing.shared_memory — Provides shared memory for direct access across processes — Python 3.9.4 documentation
From my reading of those docs, you could either create a block in python, or attach to an existing shared memory block. So presumably, one could create two MBS shared memory blocks - one to put the raw frame buffer into (from within Xojo), and one that the python script writes the result to. Both blocks would be created within Xojo using your plugin, and the names of the blocks passed to the python script. Does that sound correct?
You should try to setup a named shared memory block in a Xojo app and then check if you can get it from python.
Since non used memory doesn’t need physical memory for backing, you can easily allocate 100 MB input and output blocks and only use as much as your images need.
I don’t know if this is of any use but there is another version of OpenCV called Emgu CV. It is a cross platform .Net wrapper for Windows, Linux, Mac OS, iOS and Android.
It can be used from several different languages, including C#, VB.NET, C++ and IronPython.
I’m not saying this is of any use, but it is another option.
This is really interesting - but how (if it does), does Xojo interact with .net on the mac? I found a blog post by Paul Lefebvre from back in 2014, but it was Windows only. We are not opposed to switching this back to Windows, so Emgu CV might be of interest there, but if I can call this directly from Xojo on the mac then that basically solves my fundamental problem.
Thanks, @Christian_Schmitz - I’ll give this a try today. It’s a bit of a clunky workaround and I’d still prefer to be able to call into OpenCV directly from Xojo either through an external library or a Xojo plugin or similar. I think there would be a fair bit of interest in this from Xojo users who may not be familiar with what OpenCV can do, and how quickly and easily it can do it.
As far as what we need from it, my own interest is limited to the machine vision portions. I am not even more than vaguely familiar with whatever else it may do. That means being able to pass it a reduced version of a picture and a definition file, or data in memory, that it can run against and offer me what it sees and where in the picture. While being able to do the training and build that file would be awesome that is not 100% necessary as I can do that with the command line tools if needed. I”m not sure if that narrows it down at all or not I have python code that does what I need on the Mac and on the raspberry pi and other Linux. It feels fairly straightforward and there is actually more code associated with the capturing and pre-processing of the images than with the passing it off to OpenCV to do the machine vision stuff.
I know that a lot of this is duplicated by the built in MacOS libraries now, and they may actually be far better but I have a bad taste in my mouth for other Mac only technologies that they are letting die on the vine and I would prefer to use one thing cross platform rather than have to write everything twice once for the Pi and once for macOS even if it runs ever so slightly better there. Though when I get into it and really understand why it’s so much better I might change my mind Like I said this is a future project that is in the laying awake in the middle of the night thinking about it rather than in active development yet.
Specifically my interests are for running on MacOS and the Raspberry Pi and for the object recognition rather than specific facial recognition or other functions. I am happy to do pre-processing of any images as needed in regular Xojo code and then pass them off to OpenCV once they are ready.
Honestly though I’m not even completely sure that would make sense to someone who really knows how to do it in OpenCV. I’ve run demos and done a bunck of experimenting but would not at all qualify as someone who understands the library and everything you can do with it.
It sounds like our needs are pretty similar. We’re doing some pattern matching and then alignment of frames based on that pattern, some color space conversions, channel merging into RGB images, scaling, and possibly some stuff like generating histograms. Some of this could be done in Xojo or various plugins, but I’d like to try to do as much as possible in OpenCV, as it makes testing (with simple python scripts outside of Xojo) much easier, and makes things a bit more portable later on should we need to re-use it in something non-xojo.
1 Like
https://documentation.xojo.com/topics/declares/calling_native_macos_apis.html
This refers to calling Native OS Objective-C APIs. Am I correct in assuming that this will also work with third party libraries written in Objective C? The reason I’m asking is that we’ve started talking to someone about the possibility of creating a plain-C API for a subset of OpenCV that we can call from within our app. He dug around a bit and suggested this:
I believe Xojo can import object-oriented code from Objective-C / Swift, so I took a quick look and did find an Objective-C wrapper for OpenCV that already exists:
GitHub - MainasuK/OpenCVBridge
It also looks like the iOS version of OpenCV includes some Objective-C support. macOS and iOS are (under the hood) the same thing now, so it should be trivial to rebuild the iOS framework to run on macOS. If you went that route, you’d have an object-oriented interface and what you want might already exist.
The github project he links to is completely undocumented, and doesn’t seem to contain all the functionality we need. Is he correct that we would be able to work with these libraries via declares in Xojo the way it’s described with native obj-c? If so, we may be approaching a relatively painless solution here.