RGBSurface.Transform for Images with Alpha Channel (new format)

General
1

Hi,
starting from version 2011R4 it’s possible to create picture with AlphaChannel (instead of the classic mask) using the new constructor that omits depth.
Despite official documentation say “RGBSurface has been updated to properly support alpha channels by taking them into account when rendering objects”, I’m experiencing a strange behavior when using RGBSurface.Transform with images with Alpha Channel.

For example consider the simple transformation that inverts an image (also reported into the docs):

Put this code into the Paint event of a canvas

[code]Sub Paint(g As Graphics, areas() As REALbasic.Rect)
// RGBSurface.Transform vs RGBSurface.Pixel for Images with AlphaChannel

#if RBVersion<2011.04 then
’ Pictures with alpha channel require version 2011R4 or later
return

#else

Dim sizeX,sizeY as integer
sizeX=me.Width/2
sizeY=me.Height/2

dim fillAlpha as integer = 0 

Dim fillColor,borderColor as Color
fillColor=rgb(0,220,220,fillAlpha)
borderColor=rgb(40,40,0)

' Create two identical images
Dim imgWithAlpha1 as picture
imgWithAlpha1=new picture (sizeX,sizeY) ' New Constructor -> will create a picture with AlphaChannel
imgWithAlpha1.Graphics.ForeColor=fillColor
imgWithAlpha1.Graphics.FillOval 1,1,sizeX-2,sizeY-2
imgWithAlpha1.Graphics.ForeColor=borderColor
imgWithAlpha1.Graphics.DrawOval 0,0,sizeX,sizeY
' Draw original image
g.DrawPicture imgWithAlpha1,0,0

Dim imgWithAlpha2 as picture
imgWithAlpha2=new picture (sizeX,sizeY) ' New Constructor -> will create a picture with AlphaChannel
imgWithAlpha2.Graphics.ForeColor=fillColor
imgWithAlpha2.Graphics.FillOval 1,1,sizeX-2,sizeY-2
imgWithAlpha2.Graphics.ForeColor=borderColor
imgWithAlpha2.Graphics.DrawOval 0,0,sizeX,sizeY
' Draw original image
g.DrawPicture imgWithAlpha2,0,sizeY


' Invert imgWithAlpha1 with Tansform
dim i,map(255) as Integer
for i=0 to 255
  map(i)= 255-i
Next
imgWithAlpha1.RGBSurface.Transform map
' Draw inverted image
g.DrawPicture imgWithAlpha1,sizeX,0

' Invert imgWithAlpha2 with RGBSurface.Pixel
dim x,y,w,h as integer
dim c as color
dim s as RGBSurface
s=imgWithAlpha2.RGBSurface
w=imgWithAlpha2.Width-1
h=imgWithAlpha2.Height-1
for y=0 to h
  for x=0 to w
    c=s.Pixel(x,y)
    s.Pixel(x,y)=rgb(255-c.red,255-c.Green,255-c.Blue,c.Alpha)
  next
next
' Draw  inverted image
g.DrawPicture imgWithAlpha2,sizeX,sizeY

#endif

End Sub
[/code]

Results for fillAlpha=0 and fillAlpha=128

As you can see the Pixel method leads to better (and correct) results but its 8-10 times slower ;-(
For other image processing algorithms (like Brightness, Contrast, RGB Shift…) the results are similar, thus the question is:
how to correctly use RGBSurface.Transform for images with alpha channel (that use transparency) ?

2

It used to be that if you provided one array, it would use the values on red, green, and blue

or you could create 3 arrays, and use the overloaded version to provide
Transform(redarray,greenarray,bluearray)

maybe there should now be a

Transform(redarray,greenarray,bluearray,alphaarray)
variant?

But try the 3 array variation anyway, see if it helps…

3

The param alphaArray not exists.
Using imgWithAlpha1.RGBSurface.Transform(map,map,map) leads to the same bad result (probably because internally is the real method called even if you pass a single array).

4

There’s something definitely wrong when there’s alpha values. My suspicion is the pre-multiplied with black value isn’t properly accounted for.

To visualize the idiosyncrasy I generated an image that’s black at top going to white at bottom and alpha 0 at left going to alpha 255 on right. In this screencapture the left side is the original pictures, bottom-right is RGBSurface.Pixel inverting (correct) and top-right is Transform inverting (weird bands).

[code]Sub Paint(g As Graphics, areas() As REALbasic.Rect)
g.ForeColor = &c00FF00
g.FillRect 0, 0, g.Width, g.Height

dim p1 As new Picture(256, 256) //generate original image
dim surf1 As RGBSurface = p1.RGBSurface
for a As integer = 0 to 255
for v As integer = 0 to 255
surf1.Pixel(a, v) = RGB(v, v, v, a)
next
next

dim p2 As new Picture(256, 256) //create copy
p2.Graphics.DrawPicture(p1, 0, 0)
dim surf2 As RGBSurface = p2.RGBSurface

g.DrawPicture(p1, 10, 10) //draw originals
g.DrawPicture(p2, 10, 280)

dim map(255) As integer //transform invert 1
for i As integer = 0 to 255
map(i) = 255-i
next
surf1.Transform(map)

dim x, y As integer, c As Color //pixelwise invert 2
for y = 0 to 255
for x = 0 to 255
c = surf2.Pixel(x, y)
surf2.Pixel(x, y) = RGB(255-c.Red, 255-c.Green, 255-c.Blue, c.Alpha)
next
next

g.DrawPicture(p1, 280, 10) //draw inverteds
g.DrawPicture(p2, 280, 280)

g.ForeColor = &c808080 //box
g.DrawRect(9, 9, 257, 257)
g.DrawRect(9, 279, 257, 257)
g.DrawRect(279, 9, 257, 257)
g.DrawRect(279, 279, 257, 257)

End Sub[/code]

5

Yes, it’s applying the Transform map to the pre-multiplied values instead of the regular, unmultiplied ones.

Replace the “//pixelwise invert 2” section with this code and it produces the same image as the Transform(map) one.

dim x, y As integer, c As Color, a, v, v2 As single, v3 As integer //pixelwise invert 2 for y = 0 to 255 for x = 0 to 255 c = surf2.Pixel(x, y) a = 1 - c.Alpha / 255 //get normalzed alpha v = c.Red * a //convert red to it's pre-multiplied value v2 = (255-v) / a //do invert on premultiplied value then unmultiply v3 = v2 mod 256 //roll overflow surf2.Pixel(x, y) = RGB(v3, v3, v3, c.Alpha) //plot next next

6

Hi Will,
I suspected a rounding issue with transparent data because a transparent pixel of an image with alpha channel is reported as &c000000FF (a black color with 100% Alpha) but your analysis is far more accurate.

Does this means that there is no way to use Transform(map) until a thing like TransformWithAlpha(map,alphaMap) will be introduced ?

7

I’d say it’s a bug, not a missing feature. The docs say…

The problem is, with Alpha Channel Pictures, internally the RGB values are stored premultiplied with black. When you get the Red Green Blue values from the pixels Color they are unmultiplied for you, you never see those premultiplied values. The bug is that those internal premultiplied values are directly used as the ‘index into the map arrays’ instead of the unmultiplied values they represent. Or something like that :slight_smile:

Until the bug is fixed (or the docs define this as expected behavior) a workaround is to use Masked Pictures.

There’s no report about this in Feedback but someone should add it (I don’t have a current license to do so).

8

Images with pre-multiplied alpha channels are not suitable for image processing.

Here is why:

http://www.einhugur.com/Html/infos/premultiplied.html

9

[quote=230430:@Björn Eiríksson]Images with pre-multiplied alpha channels are not suitable for image processing.

Here is why:

http://www.einhugur.com/Html/infos/premultiplied.html[/quote]

And unfortunately it’s the only format that CoreGraphics deals with for bitmap contexts.

10

According to Apple Docs
https://developer.apple.com/library/mac/documentation/GraphicsImaging/Conceptual/drawingwithquartz2d/dq_images/dq_images.html
it seem that the is a way to use also not premultiplied formats:

[quote]When you create an image using the function CGImageCreate, you supply a bitmapInfo parameter, of type CGImageBitmapInfo, to specify bitmap layout information. The following constants specify the location of the alpha component and whether the color components are premultiplied:

  • kCGImageAlphaLast—the alpha component is stored in the least significant bits of each pixel, for example, RGBA.
  • kCGImageAlphaFirst—the alpha component is stored in the most significant bits of each pixel, for example, ARGB.
  • kCGImageAlphaPremultipliedLast—the alpha component is stored in the least significant bits of each pixel, and the color components have already been multiplied by this alpha value.
  • kCGImageAlphaPremultipliedFirst—the alpha component is stored in the most significant bits of each pixel, and the color components have already been multiplied by this alpha value.[/quote]
11

It doesn’t work for CGBitmapImageContext. See Quartz 2D Programming Guide’s Supported Pixel Formats section.

12

Ok, but I’m curious:
if both statements are true:

  • “Images with pre-multiplied alpha channels are not suitable for image processing”
    AND
  • “And unfortunately it’s the only format that CoreGraphics deals with for bitmap contexts.”
    How CG and CI handle images filtering effects that make pixel level manipulation ?
    There is a way co convert between formats?