Optimize speed of Picture to RGBA MemoryBlock algorithm

Yes, both methods produce a MemoryBlock, so you can run both on the same picture and compare the results.

The easiest way to do that is StrComp:

if StrComp( mb1, mb2, 0 ) <> 0 then
  // Didn't match
end if

Ok, now I’m with you… I didn’t think of doing that, will add something to the code to compare the blocks.

I have this niggling memory of something about color handling that Joe mentioned
Ah yes https://forum.xojo.com/2046-colors-in-cocoa might be relevant
See Joe Ranieri’s post Jun 15

[quote=36876:@Norman Palardy]I have this niggling memory of something about color handling that Joe mentioned
Ah yes https://forum.xojo.com/2046-colors-in-cocoa might be relevant
See Joe Ranieri’s post Jun 15[/quote]
I wondered about this too and just out of curiosity I read through Stuart’s code a few times to see if I could spot any issue but it seems that in both cases the image is read from a file into a Picture and then the contents of the image are read out from the Picture into Alwyn’s target. To be able to sensibly apply any colour correction to the image it would need a graphics context such as a screen or printer and so I didn’t think that colour correction would come into play until the image is rendered on screen or printed.

Some images have color profiles embedded or referenced within their files and I guess they could be used to apply corrections prior to rendering in a context (not the best idea though) but again as the same Picture class is used to load the image the treatment should be consistent whether using the pure Xojo code or the Declares approach.

Alwyn, if you want to post both pure Xojo and Declares code up here then we (forum members) can collectively review it to see if we can spot any issues. Stuart’s way of converting ARGB to RGBA is particularly neat from an optimisation viewpoint because he intentionally starts at the end of the array (+1) and moves towards the beginning moving the [A] byte to avoid doing a swap with an extra local variable and then offsets the memory by +1. I would check over that first.

HTH

I’ll post the routines, as soon as I get to my machine that has the code on it.

PURE XOJO:

Sub Constructor(texture As Picture)
  ' www.Xojo3D.com
  
  #pragma DisableBackgroundTasks
  #pragma NilObjectChecking
  #pragma StackOverflowChecking
  
  Dim x As Integer
  Dim y As Integer
  Dim offset As Integer
  Dim textCol As Color
  Dim textMaskCol As Color
  Dim w As Integer
  Dim h As Integer
  Dim rgbMain As RGBSurface
  Dim rgbMask As RGBSurface
  Dim RGBABitmapPtr As Ptr
  
  ' convert pictures to raw formats
  
  Width = texture.Width
  Height = texture.Height
  
  RGBABitmap = new MemoryBlock(Height * Width * 4) ' create a MemoryBlock for the OpenGL RGBA format
  RGBABitmapPtr = RGBABitmap
  
  ' loop through all the pixels of the picture
  
  offset = 0
  
  h = Height - 1
  w = Width - 1
  rgbMain = texture.RGBSurface
  rgbMask = texture.Mask.RGBSurface
  
  for y = 0 to h
    
    for x = 0 to w
      
      ' read the values of the current pixel
      
      textCol = rgbMain.Pixel(x,y) ' get the color of the current pixel
      textMaskCol = rgbMask.Pixel(x, y) ' get the mask (alpha) color of the current pixel
      
      ' store the color and alpha values into our OpenGL texture bitmap
      
      RGBABitmapPtr.Byte(offset) = textCol.Red
      RGBABitmapPtr.Byte(offset + 1) = textCol.Green
      RGBABitmapPtr.Byte(offset + 2) = textCol.Blue
      RGBABitmapPtr.Byte(offset + 3) = 255 - textMaskCol.Red
      
      offset = offset + 4 ' move to the next pixel in our OpenGL texture bitmap
      
    next x
    
  next y
  
  ' load texture into OpenGL
  
  OGLName = X3_LoadRGBATexture(RGBABitmap, Width, Height)
  
End Sub

WITH DECLARES:

Sub Constructor(pic As Picture)
  dim w As integer = pic.Width
  dim h As integer = pic.Height
  dim byteCount As integer = w * h * 4
  
  //=========== declare to underlying bytes
  soft declare function CGImageGetDataProvider lib "ApplicationServices" (img As Ptr) As Ptr
  soft declare function CGDataProviderCopyData lib "ApplicationServices" (prov As Ptr) As Ptr
  soft declare sub CFDataGetBytes lib "ApplicationServices" (CFDataRef As Ptr, range As CFRange, m As Ptr)
  soft declare sub CFRelease lib "CoreFoundation" ( obj as ptr )
  
  dim cgImage as Ptr = pic.CopyOSHandle(Picture.HandleType.MacCGImage) //get handle (needs release)
  
  dim dataProv As Ptr = CGImageGetDataProvider(cgImage) //get provider (dont release)
  
  dim dataRef As Ptr = CGDataProviderCopyData(dataProv) //get dataref (needs release)
  
  dim range As CFRange //range of bytes to read
  range.location = 0
  range.length = byteCount
  
  dim mem As new MemoryBlock(byteCount + 1) //make destination memory (+1 byte more)
  
  CFDataGetBytes(dataRef, range, mem) //read range of bytes into memory
  
  CFRelease(cgImage) //release those ptrs
  CFRelease(dataRef)
  
  //========== mem is now in ARGB format, scan A to other side for RGBA (why mem is 1 byte more)
  dim memPtr As Ptr = mem
  dim idx1 As integer = byteCount
  dim idx2 As integer = idx1 - 4
  while idx2 >= 0
    memPtr.Byte(idx1) = memPtr.Byte(idx2)
    idx1 = idx2
    idx2 = idx2 - 4
  wend
  
  //========= return MemoryBlock that's shifted over 1 byte so it skips that first unused A
  OGLName = X3_LoadRGBATexture(Ptr(Integer(memPtr) + 1), Width, Height)
  
End Sub

Hi,

I hope someone if watching this thread.

I need to compare two pictures and thought to try the above code with the memoryblock Alwyn posted.

I had to change the

with this code

[quote]dim textureMask as Picture=texture.CopyMask

rgbMask =textureMask.RGBSurface[/quote]

in order to be able to deal with the new picture constuctor.

So, I convert the pics to memoryblocks and then to strings and compare the strings.

Now the comparison between pictures works perfectly but I get a weird behavior: The main window resizes whenever the method is executed which I believe means that the ptr messes things up. Can anyone confirm this?

And any workaround, please?

I’ve try it on Windows 7 with GDI+ enabled

Thanks

John

Not sure about the strange behavior, but if you only need to compare the pictures, you might look at picture.GetData . It’s quite a bit faster.

I have to concur with Jim that picture.GetData might be better suited for comparing pictures.

The code posted above is a “workaround” to convert pictures into a RGBA format that can be used for OpenGL texture loading, and might not be fast enough for picture comparisons.

Hmmm, it is unlikely that the code affects that resizing of the main window, since the method is designed to run decoupled (independent) from any controls. I suspect something else might be causing this?

Hi,

Jim I will check this and compare the speed.

Alwyn, I am not really sure what is wrong wih the code.

Please check here: https://www.cubby.com/pl/Xojo%20Shared/_a10d58abd44e4d56817bb5d977100df4

Same behavior on Windows and Mac.

Thank you both.

Ok, found the problem… the original method was part of a class that had Width and Height properties.

Since the method is now implemented as a method of the window, these class properties are missing, and defaults to the Width and Height of the Window. Here is the fixed code:

[code] #pragma DisableBackgroundTasks
#pragma NilObjectChecking
#pragma StackOverflowChecking

Dim x As Integer
Dim y As Integer
Dim offset As Integer
Dim textCol As Color
Dim textMaskCol As Color
Dim w As Integer
Dim h As Integer
Dim rgbMain As RGBSurface
Dim rgbMask As RGBSurface
Dim RGBABitmapPtr As Ptr
Dim RGBABitmap as MemoryBlock

’ convert pictures to raw formats

RGBABitmap = new MemoryBlock(texture.Height * texture.Width * 4) ’ create a MemoryBlock for the OpenGL RGBA format
RGBABitmapPtr = RGBABitmap

’ loop through all the pixels of the picture

offset = 0

h = texture.Height - 1
w = texture.Width - 1
rgbMain = texture.RGBSurface

dim textureMask as Picture=texture.CopyMask

rgbMask =textureMask.RGBSurface

for y = 0 to h

for x = 0 to w
  
  ' read the values of the current pixel
  
  textCol = rgbMain.Pixel(x,y) ' get the color of the current pixel
  textMaskCol = rgbMask.Pixel(x, y) ' get the mask (alpha) color of the current pixel
  
  ' store the color and alpha values into our OpenGL texture bitmap
  
  RGBABitmapPtr.Byte(offset) = textCol.Red
  RGBABitmapPtr.Byte(offset + 1) = textCol.Green
  RGBABitmapPtr.Byte(offset + 2) = textCol.Blue
  RGBABitmapPtr.Byte(offset + 3) = 255 - textMaskCol.Red
  
  offset = offset + 4 ' move to the next pixel in our OpenGL texture bitmap
  
next x

next y

Return RGBABitmap[/code]

Notice that I replaced Width and Height with texture.Width and texture.Height respectively.

but of course…I should have spotted this.

Many thanks Alwyn for your help.

Here’s my first try at a Windows (Win32) friendly version that’s intended for use with the new-style GDI+ 32 bit pictures which are stored internally as ARGB:

Function PictureToARGBDataFast(p as picture) as MemoryBlock
#if TargetWin32
      // gets the data in ARGB format
      // we take the HBITMAP, back-convert it to a GDIPlus Bitmap object
      // then use LockBits, providing our own buffer, to receive the data
      
      // see http://msdn.microsoft.com/en-us/library/windows/desktop/ms533971(v=vs.85).aspx
      declare function GdipBitmapLockBits lib  kLibGdiPlus (bitmap as ptr, rect as Ptr, flags as uint32, pixelFormat as integer, bitmapData as Ptr) as integer
      declare function GdipBitmapUnlockBits lib  kLibGdiPlus (bitmap as ptr, bitmapData as ptr) as integer
      declare function GdipCreateBitmapFromHBITMAP lib kLibGdiPlus (hbitmap as ptr, hpalette as ptr, GpBitmapPtr as Ptr) as integer
      declare function DeleteObject lib kLibGDI32 (handle as Ptr) as boolean
      
      dim hbitmap as Ptr = p.CopyOSHandle(picture.HandleType.WindowsBMP)  // must remember to call DeleteObject when done
      dim mbBitmapPtr as new MemoryBlock(4)
      dim r as integer = GdipCreateBitmapFromHBITMAP(hbitmap,nil, mbBitmapPtr)  // converts our HBITMAP back to a GDI Plus Bitmap object
      dim bitmapPtr as Ptr = mbBitmapPtr.ptr(0)
      
      dim mb as new MemoryBlock(p.width*p.height*4)
      
      // see http://msdn.microsoft.com/en-us/library/windows/desktop/ms534137(v=vs.85).aspx
      const imageLockModeRead = &h0001
      const imageLockModeWrite = &h0002
      const imageLockModeUserInputBuf = &h0004
      dim flags as integer = imageLockModeRead + imageLockModeUserInputBuf
      
      //  see http://msdn.microsoft.com/en-us/library/cc230858.aspx
      const PixelFormat32bppARGB = &h0026200A  // non-premultiplied
      const PixelFormat32bppPARGB = &h000E200B // premultipliedf
      dim pixelFormat as integer =  PixelFormat32bppARGB
      

      // note: define RECTANGLE as a Structure with 4 integers: left,top,right,bottom
      dim rect as RECTANGLE 
      rect.left = 0
      rect.top = 0
      rect.right = p.Width
      rect.bottom = p.height
      dim mRECT as MemoryBlock = rect.stringValue(true)  // copy from structure to memoryblock
      
      dim bmd as BitmapData
      bmd.width = me.width
      bmd.height = me.height
      bmd.pixelFormat = pixelFormat
      bmd.stride = me.width*4
      bmd.scan = mb  // set the pointer to the data buffer
      dim mBMD as MemoryBlock = bmd.StringValue(true)  // copy the sturcture to the memoryBlock
      
      r = GdipBitmapLockBits(bitmapPtr, mRECT, flags, pixelFormat, mBMD)
      bmd.StringValue(true) = mBMD  // copy memory block back to structure for debugging to see if anything changed
      // clean up
      call GdipBitmapUnLockBits(bitmapPtr, mBMD)
      call DeleteObject(hbitmap)
      return mb
#endif

Notes:

  • I’m not sure how fast this actually is - the line where we call GdipCreateBitmapFromHBITMAP(), I suspect the OS is creating yet another copy of the picture data, so this routine may actually end up making 3 copies of the data before it’s done. Needs some benchmarking.

Well, I think you make several copies and I try to edit the original image data, so I need pointer.

see also
https://forum.xojo.com/13040-gdipbitmaplockbits

While working with Alain Bailleul on the Xharity project, he came up with a very elegant solution to convert a Picture object to raw BGRA data.

Not not only is his way much less code, I’m getting 200%+ speed increases with his method.

Sub Constructor(texture As Picture, transparent As Boolean = False)

[code] Dim mb as MemoryBlock

’ get the pixel data, but as it is in BMP uncompressed format it’s flipped upside down and in BGRA format
’ the first 54 bytes are the header, so remove them. the rest is BGRA data

mb = texture.GetData(Picture.FormatBMP)
BGRABitmap = mb.StringValue(54, Height * Width * 4)

TextureImage = texture
OGLName = 0[/code]

Thought I’d share it here for those who might be interested in a fast way to converting Picture objects into raw memoryblock pixel data.

Some irrelevant code in the example, so here is cleaner version:

Dim mb as MemoryBlock

' get the pixel data, but as it is in BMP it is flipped upside down and in BGRA format
' the first 54 bytes are the header, so remove them. the rest is BGRA data
  
mb = texture.GetData(Picture.FormatBMP)
BGRABitmap = mb.StringValue(54, Height * Width * 4)

BGRABitmap is a MemoryBlock.

[quote=36392:@Kem Tekinay]Use a Ptr instead of the MemoryBlock for extra speed.

dim RGBABitmapMB as new MemoryBlock(Height * Width * 4) ' create a MemoryBlock for the OpenGL RGBA format
dim RGBABitmap as Ptr = RGBABitmapMB

MB.Byte is a function call, but Ptr.Byte is a peek/poke. Use with caution.[/quote]
Kem, can you please explain in detail the exact advantages and disadvantages, as well as the difference between Pointers and memoryBlog? Why are pointers faster?

A pointer is a C-thingy. Whenever I tried to learn C I had to give up when using a pointer. Awful stuff.

The memoryblock occupies a location in the RAM. The pointer points (sic!) to the address of the memoryblock. If you have the pointer you can talk to the data that the pointer points to. You can quite easily get data that doesn’t belong to the memoryblock if you talk to an address after the memoryblock ends. That’s what “peek/poke - use with caution” means.

Does that make sense?