I’ve been racking my brain on the fastest possible way to convert a normal Xojo Picture (including its alpha mask) into an RGBA MemoryBlock that holds the raw data that OpenGL understands when loading Bitmap textures. After pulling some code from the old Real Studio forums a while back, and making some modifications, this is what I ended up with:
Function ConvertToRGBA(texture As Picture) As MemoryBlock
#pragma DisableBackgroundTasks
#pragma NilObjectChecking
#pragma StackOverflowChecking
Dim x, y, offset As Integer
Dim textCol As Color
Dim textMaskCol As Color
Dim alpha As Byte
' convert pictures to raw formats
Width = texture.Width
Height = texture.Height
RGBABitmap = new MemoryBlock(Height * Width * 4) ' create a MemoryBlock for the OpenGL RGBA format
' loop through all the pixels of the picture
offset = 0
for y = 0 to Height - 1
for x = 0 to Width - 1
' read the values of the current pixel
textCol = texture.RGBSurface.Pixel(x,y) ' get the color of the current pixel
textMaskCol = texture.Mask.RGBSurface.Pixel(x, y) ' get the mask (alpha) color of the current pixel
' calculate the OpenGL alpha values, using the mask values of the pixel
alpha = 255 - textMaskCol.Red
' store the color and alpha values into our OpenGL texture bitmap
RGBABitmap.Byte(offset) = textCol.Red
RGBABitmap.Byte(offset + 1) = textCol.Green
RGBABitmap.Byte(offset + 2) = textCol.Blue
RGBABitmap.Byte(offset + 3) = alpha
offset = offset + 4 ' move to the next pixel in our OpenGL texture bitmap
next x
next y
return RGBABitmap
End Function
Problem is, this is still not fast enough when working with very very large pictures.
I was wondering if anyone can spot anything that I could change to speed up this algorithm to map a normal Picture object to an RGBA MemoryBlock?
textCol = texture.RGBSurface.Pixel(x,y) ’ get the color of the current pixel
textMaskCol = texture.Mask.RGBSurface.Pixel(x, y) ’ get the mask (alpha) color of the current pixel
you’d better cache those RGBsurface objects in local properties.
But at some point, we can only optimize by providing a plugin function for this.
If you use the MBS plugins, you can do this in one (or two) lines, and going to be much faster than you could ever get using Xojo code. Plus, MBS has functions to scale & resize pictures using multiple background threads. I’ve done extensive benchmarking on this, and using MBS is the way to go.
My code looks something like this:
// p is a picture, dataRGB is the destination memoryblock
call p.CopyRGBXtoMemoryblockMBS(dataRGB,0)
if p.mask(false) <> nil then
' picture has a mask -- get Alpha data -- since alpha pixels have R = G = B, we just arbitrarily copy only the B component
call p.mask.CopyBtoMemoryblockMBS(dataRGB, 3, 4)
end if
This doesn’t work with new-style pictures on Win32, but Christian is aware of it. It works fine with old-style pictures.
Just an idea (that I have not tried!) I wonder if walking along the data from the MemoryBlock returned by Picture.GetData() would be faster than lots of RGBSurface.Pixel() calls? Picture.GetData() looks like it returns an in-memory image file so for a TIFF file you would get a TIFF header and some Tags to navigate around the file and some data in the header to tell you how to interpret the image data. It is not simple to write the code to interpret the image file’s content but you could simulate a speed test by just using any data in the MemoryBlock to see if it speeds things up and then if it does you can invest the time to work out the exact layout of the image. In fact even before you do that, just comment out the Pixel() calls and see how much time they take.
If you go ahead then you will need to know the format of the image file that is going to be returned by GetData() and if you are looking to preserve the detail and transparency then you could start with the TIFF, PNG and BMP files to find one that hopefully gives RGBA (RGB should be easy but RGBA might be harder to find support for - it will depend on Xojo’s implementation).
That combined with Kem’s Ptr suggestion and avoiding incrementing by more than one and extra operations (kind of what Dave suggested) could get you something.
The Xojo language reference is a bit unclear about RGB Surface, on one hand it says that ‘It is extremely fast faster than accessing pixels through the Graphics property’ but two sentences back it says ‘Calling RGBSurface is a computer-intensive process’ so I don’t doubt that it is faster than accessing pixels via Graphics but when I build a colour chart in one Xojo app and I have not found it to be as fast as I would wish for.
Picture.FormatBMP (not supported in Console apps)
Picture.FormatGIF (not supported in Console apps)
Picture.FormatJPEG
Picture.FormatPNG
Picture.FormatTIFF (not supported in Console apps)
Are you saying that the other formats are not supported?
BMP supports RGBA but I do not know if ‘Xojo BMP’ supports RGBA, there is also RGBA in PNG and TIFF from my recollection, again I don’t know if Xojo supports them. Maybe Norman can comment?
First step as I suggested above should be to comment out the .Pixel() calls and see if the performance is adequate otherwise it is not good use of time.
Second step could be to just read anywhere (safely) in the memory block just to simulate a read and check the performance again.
If it looks good then you can go about decoding the file format that works (i.e. gives you RGBA or RGB if sufficient)
First I did a profile run with the method prior to optimization, and the method took an average of 177ms on my test picture. Then I modified the code as follow:
#pragma DisableBackgroundTasks
#pragma NilObjectChecking
#pragma StackOverflowChecking
Dim x As Integer
Dim y As Integer
Dim offset As Integer
Dim textCol As Color
Dim textMaskCol As Color
Dim w As Integer
Dim h As Integer
Dim rgbMain As RGBSurface
Dim rgbMask As RGBSurface
Dim RGBABitmapPtr As Ptr
' convert pictures to raw formats
Width = texture.Width
Height = texture.Height
RGBABitmap = new MemoryBlock(Height * Width * 4) ' create a MemoryBlock for the OpenGL RGBA format
RGBABitmapPtr = RGBABitmap
' loop through all the pixels of the picture
offset = 0
h = Height - 1
w = Width - 1
rgbMain = texture.RGBSurface
rgbMask = texture.Mask.RGBSurface
for y = 0 to h
for x = 0 to w
' read the values of the current pixel
textCol = rgbMain.Pixel(x,y) ' get the color of the current pixel
textMaskCol = rgbMask.Pixel(x, y) ' get the mask (alpha) color of the current pixel
' store the color and alpha values into our OpenGL texture bitmap
RGBABitmapPtr.Byte(offset) = textCol.Red
RGBABitmapPtr.Byte(offset + 1) = textCol.Green
RGBABitmapPtr.Byte(offset + 2) = textCol.Blue
RGBABitmapPtr.Byte(offset + 3) = 255 - textMaskCol.Red
offset = offset + 4 ' move to the next pixel in our OpenGL texture bitmap
next x
next y
This brought down the average run time to 52ms on my test picture (an performance boost of 70%). The most significant improvement was due to Christian’s suggestion to cache the RGBSurfaces (accounted for about 110ms of the savings).
All the other suggestions added a compounded improvement to the overall performance.
The GetData() method will involve a lot of extra work, so for now I’m happy with the new optimized version of the method.
Thanks for the links to the MBS plugins Christian, I’ll have a loot at that.
I did this as you suggested Carl, and the run time went down to 10ms, so there is definitely merit in eventually looking at a GetData() improvement, perhaps using the BMP format, since that seems to be the easiest format to parse. But it will involve some work to get it done right.
I just want to point out, this PicToRGBA method is part of an open source Xojo 3D engine I’m working on, and I’d like to avoid the use of plug-ins if possible, and keep it pure Xojo. But I can definitely see that the MBS plugins might be very useful in some of my other projects.
When you are ready for it, the information on Wikipedia is quite illustrative and there is lots of sample code on DIBs and BMP files all the way back to the Petzold days. A few structures in Xojo should help but it is not trivial (nor probably necessary) to implement all of the BMP options.
The test picture wasn’t that large, 512x512, but below is a screenshot of a 3D model that uses 38 bitmap textures, ranging from 640x640 to 2000x2000… so it was more a concern of a large number of pictures, rather than one single picture being very large.
The model used to take about 30s to load, but it now loads at around 8s using the new improved algorithm (I am now using your suggestion to use Ptr’s to access MemoryBlocks in other parts of the code as well… with great success in improving the runtime speeds).
