The future of high performance graphics in Xojo

Hi guys,

A number of threads over the years have raised concerns about graphics performance in Xojo. Specifically I’m talking about the use of the canvas to perform either image manipulation or game development. I was sad when Xojo deprecated and removed the SpriteSurface and the subsequent SuperSpriteSurface died.

Those of us who tinker with game development in Xojo are well aware of the limitations of the Canvas control and many of us have moved to using OpenGL or the (now deprecated / not maintained) ezgl control (https://ezgl.tumblr.com). Whilst Xojo do provide a (barebones) control for OpenGL - OpenGLSurface (http://documentation.xojo.com/api/graphics/opengl.htmlSurface), those of us on macOS know that OpenGL’s days are numbered as it has been formally deprecated in favour of Apple’s proprietary Metal API. This is bad news if you have any aspiration of making a game or graphics-intensive app with Xojo on macOS.

Certainly in the Windows, Android and Linux world, the future of graphics is Vulkan (Vulkan - Wikipedia. Think of this as a “new” OpenGL. It is also available on macOS and iOS through the open source and formally supported (financially by Valve) MoltenVK C++ library (GitHub - KhronosGroup/MoltenVK: MoltenVK is a Vulkan Portability implementation. It layers a subset of the high-performance, industry-standard Vulkan graphics and compute API over Apple's Metal graphics framework, enabling Vulkan applications to run on macOS, iOS and tvOS.). This allows you to write graphics code using Vulkan and it automatically bridges it to Metal.

What I would like to know is whether there is any interest on the forum in creating a VulkanSurface control? All this would do is provide the bare basic Vulkan context that can be drawn to. It would need to be implemented as a plugin (like the OpenGLSurface) and would have to somehow statically link to the MolktenVK library to allow the control to work on macOS.

This is beyond my skill set but I have a real need for a high performance surface to draw to for the current personal project I’m working on. I’m finishing up an open source 2D physics engine and rendering a 2D game is just to slow using Xojo’s native Canvas control and I don’t want to invest in the OpenGLSurface control as I strongly suspect OpenGL will be dead come macOS 10.16.

I’m specifically asking @Christian Schmitz, @Björn Eiriksson, @Sam Rowlands and @Norman Palardy if they have any interest in developing this? Depending on the cost I might be able to fund the initial development myself or perhaps we could look to Patreon or crowdfund it’s development.

I just think that having a high performance graphics option is important to Xojo and I feel strongly that Xojo could be a good platform for 2D game creation (especially when I eventually get my physics engine out there!) but it needs a decent canvas option.

Thoughts?

Definitely. We have a client project that is using OpenGL in their product and we know we’ll need to replace it with something in the near future. I can probably get my client to do some funding.

I like the idea of crowdfunding because I don’t see Xojo doing it. And it might be quite expensive to get the initial port working and to the point where it’s useful for most Xojo developers.

I have worked with projects in Windows with Vulkan and DirectX 11. Vulkan is seriously complicated, and it takes about 1,000 lines of code to draw a rotating box. Granted most of this is setting up the proper instances in Vulkan. DirectX 11 is Windows only, and is much easier to program and has very good performance. DirectX 12 is Vulkan-ish and is getting complicated to program.

OpenGLSurface in Xojo has quite a few bugs in the Windows version. It’s a nice safe way to learn performance graphics, and advanced graphics has limitations. The Windows version of OpenGL appears to be about 1.3.

Although your question is about Macs, I would be interested in a VulkanSurface control for Windows OS, or even DirectX 11.

I’m looking for a cross platform solution as I need to target at least macOS and Windows, hence why I’m concerned about OpenGL. 1000 lines of code to get a cube on the screen is a depressing thought but if that’s the future then so be it.

I would suppose that the majority of Xojo developers are more interested in rendering 2D images than 3D surfaces so I wonder whether there is scope for a GPU accelerated canvas to help with this?

Perhaps something akin to Tin Rocket’s old SuperSpriteSurface but without the extraneous collision detection, etc. I believe that was backed by an OpenGLSurfsce but abstracted away a lot of the OoenGL commands.

It is true that I only use it to get Core Image processed images displayed on screen as quickly as possible, I was forced to go down this route because Apple deprecated & broke the system I was using in 10.10. So I moved to OpenGL (which took ages to get the settings right), only to find that in 5 years that’s dead too and now I have to use Metal (which isn’t upto OpenGL in my experience).

There are others I know of who use it for actual 3D rendering, I’ll privately poke them to see if they’ll join in.

If you are building a casual game, you can do it with the Xojo canvas and judicious use of invalidate( left, top, width, height), as that’s what we did back in 2005 for a casual game “Recyclorama” that my wife and I made. I also used CGLayers on the Mac (as they were blindingly fast), but that’s what Apple broke and deprecated in 10.10.

Is Vulkan self-contained or does it require libraries to be installed on the users computer first?

I had to install runtime libraries for Vulkan to work on Windows 10. The Vulkan SDK has packages and runtime libraries installed with the package. A few of the newest games that I downloaded over the holidays automatically installed Vulkan libraries.

Window DirectX 9.0 required downloaded libraries to run, and DirectX 11 and 12 are automatically installed in Windows.

Sorry, I am not sure about Mac.

Edit: The Vulkan SDK is also for Linux.

I suspect it would be the same. There is one competing application to our HDRtist, it’s over 1GB in file size, while ours is only 50mb. As their app runs on Mac and Windows, I suspect that they’re using an x-plat library for hardware acceleration.

I think the Vulkan libraries are installed as part of the graphics card driver install. It’s supported by most modern GPUs.

In order to get Vulkan to run on macOS, the user would need the MoltenVK library installed (as this is a lightweight translation layer to Metal). That can be either a dynamic or statically linked library.

Well, as a plugin vendor I can decide to start a new project.

I just spend a few weeks on Image Magick 7 APIs for a client project and got a new plugin. Try it.
We also spent a lot of time this year on SceneKit from Apple to do 3D graphics and animation.

If someone proposes a C library to interface via plugin, we could create such a plugin. But please not just say to wrap Vulkan as this is huge and we still need something for Mac. Best would be to interface a library which does internally wrap to several APIs.

@Christian Schmitz How about either of these two as cross platform wrappers? Could you create a canvas control utilising either of these?

https://github.com/DiligentGraphics/DiligentEngine

https://github.com/bkaradzic/bgfx