I think we had a miscommunication. I wasn’t suggesting combining the two solutions, but using the example of the file-based solution to ask if the same technique would be possible with Shared Memory on the backend instead of a file, and if a View would already do what my proposed wrapper class (that emulates a Double array) would do.
I see, in that case I believe yes. You could build whatever additional capability you wish on top of these SharedMem and SharedView classes, and have them operate via shared memory rather than file-based, transparently. A good idea.
Thank you Christian for your 15 min turnaround time after I requested this (I don’t know how you do it!). Although MBS can now open Views that are >2GB, Christian and I then uncovered a bug in the latest xojo release whereby Ptr.anything() crashes as soon as you exceed a 2GB index. Christian reported the nuts and bolts of why in a separate post and the feedback report.
Pls sign on to the bug report to get this fixed ASAP:
2 Likes
The code for the mo program and the CI is owned by the University of CT. You are welcome to a copy if you sign a non-disclosure agreement. I do not sell the programs, but distribute them freely. The code however involves a number of the faculty, and for some reason, UConn thinks it is theres.
If you think my input could be helpful, I’d be happy to sign that.
Turns out this was known and reported since April 2020:
<https://xojo.com/issue/59705>
Kudos at least, to Xojo for fixing it today, the same day Christian reposted the bug.
It would be very interesting to hoist the matrix of eigenvectors onto the GPU and then use Metal Compute to produce the result in parallel. I’ve done this with 1920x1080 pixel data (and shared memory with the CPU).
The two GPU’s I’ve had experience with to date are the iMac 2015 w/Radeon R9 M395 (28 compute cores) and the iMac 2019 w/Vega 48 (48 compute cores).
Hi Bruce,
This is very interesting, GPU is something I’ve always wanted to harness. Is this difficult to do? Do you have some example code? I assume this would all have to be done via plugins written in C or some such?
From Xojo, a plugin would be needed. My experience is otherwise writing in Objective C (on Mac). I’d be happy to help if you were using a Mac and I had some idea of what the data requirements are… eg. are these static eigenvectors encapsulated by a 200x200 matrix of 64-bit floats? The matrix elements would need to reside in shared memory. We’d need to figure out the parameters for the plugin call.
The Metal Compute stuff I’d use would come from here, with the juicy stuff around page 137:
Thx Bruce, I’ll have a look. My data are image data like yours, multidimensional matrices of UInt16s.
I saw this post today by John Carmack (of “Doom” fame). There’s some pretty powerful stuff out there in this space – that even eclipses “numPy” on Python.
There’s some craziness here on the Mac side I just haven’t had the time to delve into:
Thanks for the offer of help Bruce, much appreciated. After briefly looking over the docs, looks very interesting, and probably exactly what I need, but the time required to get this to work is something I fear I will never have enough of. Multicore with console apps, something I’ve figured out and had working for several years now, will have to do. And the Worker class will make it that much easier.