Picture Differences Console vs Desktop due to AlphaChannel

Targets macOS
21

Opening a picture on Desktop Mac is shifting the reported rgb values but the actual values seem to still be there.

This is my simple test
Create and fill a Picture with RGB(255, 128, 64)
In both Console and Desktop they report back 255, 128, 64
Save that picture then Open it
Now Console reports 255, 128, 64 but for me Desktop reports 251, 129, 63

To get the original values in Desktop I used some classes that wrap the CG functions.

dim p As Picture = Picture.Open(SpecialFolder.Desktop.Child("orange.png")) dim xp As new XPicture(p) //creates a CGBitmapContextRef from Picture dim data As Ptr = xp.data //get Memoryblock of the underlying pixels log( Str(data.UInt8(0)) + ", " + Str(data.UInt8(1)) + ", " + Str(data.UInt8(2)) ) //reports 255, 128, 64

My classes aren’t tested on 64bit and it’s clumsier to access pixels in a Memoryblock and any premultiplication isn’t handled for you. A benefit is that access is much faster than RGBSurface.

Here’s an outline of the steps
-Starting with the Picture, use it’s handle to get a CGImage.
-Create a CGBitmapContextRef the same size, with Device/Generic color space, and provide a Memoryblock for the pixel data store
-draw the CGImage into the CGBitmapContextRef
-The Memoryblock now contains your pixel values.

Here’s a trimmed down version with just the functionality you need to show the process of getting 8bit RGBA pixels. There’s other places you can find code for doing this like MacOSLib, or anything that talks about CG/Core Graphics/Quartz 2D, and there’s probably other avenues through the CG functions. Here’s the confusing docs if you want to dive in.

[code]//all added to class XPicture
Private Const daLib = “CoreGraphics”
Private mBitmapPtr As Ptr
Private mData As MemoryBlock
Private mHeight As Integer
Private mWidth As Integer

Sub Constructor(pic As Picture)
declare sub CGContextDrawImage lib daLib (cntxt As Ptr, r As XRect32, img As Ptr)
declare sub CGImageRelease lib daLib (id As Ptr)

Constructor(pic.Width, pic.Height) //create data and bitmapPtr

dim cgimg As Ptr = pic.CopyOSHandle(Picture.HandleType.MacCGImage)

dim r As XRect32
r.x = 0
r.y = 0
r.w = pic.Width
r.h = pic.Height

CGContextDrawImage(mBitmapPtr, r, cgimg)

CGImageRelease(cgimg)

End Sub

Sub Constructor(pwidth As integer, pheight As integer)
declare function CGColorSpaceCreateDeviceRGB lib daLib () As Ptr
declare sub CGColorSpaceRelease lib daLib (id as Ptr)
declare function CGBitmapContextCreate lib daLib ( _
data As Ptr, width As Integer, height As Integer, _
bitsPerComponent As Integer, bytesPerRow As Integer, _
space As Ptr, bitmapInfo As UInt32) As Ptr

mWidth = pwidth
mHeight = pheight

mData = new MemoryBlock(mWidth * mHeight * 4)

dim bitsPerComp As UInt32 = 8
dim bytesPerRow As UInt32 = 4 * mWidth
dim bitmapInfo As UInt32 = 16385 //4 << 12 | 1, kCGBitmapByteOrder32Big | kCGImageAlphaPremultipliedLast
dim space As Ptr = CGColorSpaceCreateDeviceRGB

mBitmapPtr = CGBitmapContextCreate(data, mWidth, mHeight, bitsPerComp, bytesPerRow, space, bitmapInfo)

CGColorSpaceRelease(space)

End Sub

Private Sub Destructor()
declare sub CGContextRelease lib daLib (id As Ptr)
CGContextRelease(mBitmapPtr)
End Sub

Function data() As MemoryBlock
return mData
End Function

Structure XRect32
x As CGFloat
y As CGFloat
w As CGFloat
h As CGFloat[/code]

edit: updated to work on 64bit (I think). Some types in CGBitmapContextCreate changed from UInt32 to Integer.

22

Koos,

If you want to have consistent results within each platform you could make your helpers also using desktop applications, only without a window. The amount of extra memory used by a desktop is negligible in modern computers.

It will not give you consistent results across platforms though.

Julen

23

Julen, nice observation and suggestion, but as you indicate, across platforms there could be a problem

Will, thanks for that code and testing. Just before your reply, I was working with the Einhuger GraphicsFormat plugin and got some interesting result:

Dim pic as Picture
  Dim jpg as JpegImporter
  Dim c as Color
  jpg = new JpegImporter
  pic = jpg.OpenFromFile(GetFileAtRoot("j.jpg"))
  if pic <> nil then
    Canvas1.Backdrop = pic
  end if

In this case the pixels reported back exactly what was shown in PS, for my test pixel. So what this does is read the data from disk bypassing Picture.Open and assigning the result to the picture object, and then drop it into the Canvas Backdrop

What I observed though was that the display colors were slight faded compared to if I just drop it into an ImageWell on Xojo, or if I viewed it under PhotoShop/Preview. I started worrying that when I add colors to this, say changes to some pixels through the Picture (g.DrawFill etc) what will happen.

I was contemplating the way forward when your suggestions came in. Interesting thet RGBSurface and the memoryblock differs

My major question at this stage was from your code you do the following:
Create a Xojo.Picture Drawing with ORANGE(255,128,64) color and save that as Orange.png
When you read it back, in console it was still ORANGE, but on the desktop it was say VividORANGE(251,129,63)
You then looked in the memoryblock of the picture an see it is still ORANGE

I did a few further experiments
I created a Orange.Png in Photoshop with RGB(255,128,64)

  dim p as Picture = Picture.Open(SpecialFolder.Desktop.Child("Orange.png"))
  p.Save(SpecialFolder.Desktop.Child("Orange1.png"), p.SaveAsPNG)

and then

  dim p as Picture = Picture.Open(SpecialFolder.Desktop.Child("Orange1.png"))
  p.Save(SpecialFolder.Desktop.Child("Orange2.png"), p.SaveAsPNG)

Now we are going to find some further interesting quirks:
In Photoshop Orange1 is RGB(252,106,50)
In Photoshop Orange2 os RGB(252,106,50)

If Xojo.Picture.Open is reading PNG created in Photoshop, it will change the RGB values if writing it out again. Orange vs Orange1
From this I can observe that if Xojo.Picture did the save, and it reads the file and save it again, it will stay the same RGB values. Orange1 vs Orange2

So I then did the same with JPG
Orange1 RGB(251,106,51)
Orange2 RGB(251,106,51)

This is rather interesting because it just shows that some translations are happening, and the translations have some rounding and possibly truncation somewhere because the translation works the same when executed a second time

BUT: If you read a non XOJO.Picture created file INTO picture, and then use the memoryblock RGB values, you will get the original values first time, but if you save it, you’ve lost the orginal values

I then started questioning what the Draw methods will do. So basically based on your Memoryblock findings I said:
Lets read the Original Photshop JPG file
Add two Ovals
Oval 1 should be same as the Original Photoshop RGB Orange value
Oval 2 should be the same as what you observed it changes to when read into Picture

  dim p as Picture = Picture.Open(SpecialFolder.Desktop.Child("Orange.jpg"))
  
  p.Graphics.ForeColor = RGB(255,128,64)
  p.Graphics.FillOval(10,10,40,40) // oval 1
  p.Graphics.ForeColor = RGB(251,106,51)
  p.Graphics.FillOval(50,50,40,40) // oval 2
  Canvas1.Backdrop = p
  p.Save(SpecialFolder.Desktop.Child("Orange2_adjusted.jpg"), p.SaveAsJPEG)

The Orange2_Adjusted brings back values as follows:
Background and Oval2 RGB(251,106,51)
Oval1 RGB(255,128,64)

for those who wonder, these two sets of Orange are rather widely different when viewed on the screen!!

So I then did the same under a console App (just excluded the Canvas1.Backdrop
Results were:
Background and Oval1 RGB(255,128,64)
Oval2 RGB(251,106,51)
As expected! Eureka!

This console part is just interesting because it is equivalent to what will happen under a PC version of the app.

The problem I see is that if you read in an image created outside your App, and then saving it, it changes the colors. For now I assume this is relevant to Mac Desktop only. So if you mix from various sources, you get a change in result.

Just to test a bit further I did a read back in Xojo of the Orange.Jpg and Orange1.jpg and inspected using RGBSurface Pixel

  dim p as Picture = Picture.Open(SpecialFolder.Desktop.Child("Orange.jpg"))
  dim result as string
  dim c as Color
  c = p.RGBSurface.Pixel(1,1)
  result = "Orange.jpg: RGB = " +c.Red.ToText + ", " + c.Green.ToText + ", " + c.Blue.ToText 
  p = Picture.Open(SpecialFolder.Desktop.Child("Orange1.jpg"))
  c = p.RGBSurface.Pixel(1,1)
  result = result + chr(10) + "Orange1.jpg: RGB = " +c.Red.ToText + ", " + c.Green.ToText + ", " + c.Blue.ToText 
  msgbox(result)

Results:
Orange.jpg RGB(252,106,51)
Orange1.jpg RGB(251,106,51)
Not sure where that extra 1 comes from either.

So I decided to use your XP Class and got the following back
Orange.jpg: XPicture RGB(255,128,65)
Orange1.jpg: XPicture RGB(254,128,65)
Not sure how that 1 jumped from the 255 to the 64, but it shows the memoryblock has values same to the original.
Here is the code for the XPicture versions

  dim p as Picture = Picture.Open(SpecialFolder.Desktop.Child("Orange.jpg"))
  dim result as string
  dim c as Color
  c = p.RGBSurface.Pixel(1,1)
  result = "Orange.jpg: RGB = " +c.Red.ToText + ", " + c.Green.ToText + ", " + c.Blue.ToText 
  dim xp as new XPicture(p)
  dim data as Ptr = xp.data
  result = result + " - " + Str(data.UInt8(0)) + ", " + Str(data.UInt8(1)) + ", " + Str(data.UInt8(2))
  
  p = Picture.Open(SpecialFolder.Desktop.Child("Orange1.jpg"))
  c = p.RGBSurface.Pixel(1,1)
  result = result + chr(10) + "Orange1.jpg: RGB = " +c.Red.ToText + ", " + c.Green.ToText + ", " + c.Blue.ToText 
  xp = new XPicture(p)
  data = xp.data
  result = result + " - " + Str(data.UInt8(0)) + ", " + Str(data.UInt8(1)) + ", " + Str(data.UInt8(2))

BUT wait, Photoshop reported RGB for Orange1.jpg as Orange1 RGB(251,106,51), whereas XPicture shows the data is still
Ok, so this is still not the end of this.
Orange.jpg was sRGB IEC61966-2.1 color profile
Orange1.jpg (generated by XOJO) was Generic RGB Profile

So it seems that XOJO when loading the picture must somehow check the color profile and adjust the underlying memoryblock pixels accordingly. So if sRGB then they get made a bit more Vivid for the display part, but stored as original. When Generic RGB, then it seems the memoryblock is scaled to less Vivid, while the display remains the same. This could explain the slight changes in the RGB codes from 64 to 65 and 255 to 254 etc.

I now need to go back and have a look at what Einhuger’s direct load does, because I suspect it ignores the Color Profile.
So It reads Orange.jpg as 255,128,65 and Orange1.jpg as 251,106,51.

This might seem different to what Will was seeing, but Will created the file with Xojo and Generic RGB to start with, and then read it back using Picture, and inspect the underlying memory block after the read. So this behavious was not apparent.

I now need to figure out what will happen under iOS. And I do not have a Pixel inspector for iOS, so lots more work to do teh testing there.

And now that I’ve found all this, I still need to understand how to work knowing all this.

24

You are bumping into the physiological perception versus the actual color. Color profiles are to the eye a bit like the “disco” setting of an audio equalizer to the ear : they give you an apparently better rendition by setting it in the color frequencies better perceived by the human eye.

The reason why it shows better in Photoshop is that the picture is shown to you with the color profile rendition. You can make the experience yourself. Reminder of the very long thread about a yellow box.

Create a new document in Photoshop, 100x100.
Create a new pure yellow color &cFFFF00
Drop it into the picture
Press Shift-Cmd-4, capture the area
Load the resulting picture, and check the value of yellow :
&cFFFF33

Photoshop works on the actual values internally, but shows them with the filter of color profile.

I believe you can do pretty much the same by using a picture saved to disk then loaded into Xojo the regular way, while doing pixel over the pure color Einhugur picture. That way you keep the most vivid rendition, while keeping the exact color available.

25

Michel

You’ve put is nicely. The Photoshop example will be good to illustrate to others

So I think it will be the same everywhere, but on the Desktop I need to have 2 copies of the picture being modified and analysed. This will be at the Einhugur level, and then shown at the Xojo level for the Vivid Colors.

Had to go through all this exploration to get an understanding.

Thanks for all the inputs. Much appreciated