Slower canvas updating on external display

MacBook Pro 2015 Retina 3,1GHz

I’m working on an app that scrolls images smoothly, using a canvas which takes up most of a main window, updated using .Invalidate from a Timer 25 times a second (so, Timer period of 40 ms). This “frame rate” was determined by trial and error as giving a smooth result and being manageable with the Xojo Timer which is limited to integer period values.

I’ve noticed a difference in this “frame rate” when the window is being displayed on the MacBook’s built-in display, it’s smooth. When the window is displayed on my external monitor (32" 1920 x 1080 @ 60 Hz) the frame rate is so slow that the drawing becomes out of sync. Simply dragging the window back and forth between the two monitors shows the difference in the drawing speeds.

If it were a matter of the refresh rate of the monitor, that would be understandable, but that doesn’t appear to be the problem. Something is actually slowing down the actual drawing. The app processes more slowly. The timer seems to fire more slowly. ? Obviously, it shouldn’t do that.

Is it happening because I’ve selected 1920 x 1080 when the default resolution for the monitor is 3840 x 2160 ? Forgive me for the dumb question which I could answer myself by changing my screen resolution. (I don’t want to change the resolution because it simply wreaks havoc with my workflow.)

Here is the monitor data from System Report:
Intel Iris Graphics 6100:

Chipset Model: Intel Iris Graphics 6100
Type: GPU
Bus: Built-In
VRAM (Dynamic, Max): 1536 MB
Vendor: Intel
Device ID: 0x162b
Revision ID: 0x0009
Metal: Supported, feature set macOS GPUFamily1 v4
Displays:
Colour LCD:
Display Type: Built-In Retina LCD
Resolution: 2560x1600 Retina
Framebuffer Depth: 24-Bit Colour (ARGB8888)
Mirror: Off
Online: Yes
Automatically Adjust Brightness: No
Connection Type: Internal
U32J59x:
Resolution: 3840x2160 (2160p/4K UHD 1 - Ultra High Definition)
UI Looks like: 1920 x 1080 @ 60 Hz
Framebuffer Depth: 24-Bit Colour (ARGB8888)
Display Serial Number: HNMN700129
Main Display: Yes
Mirror: Off
Online: Yes
Rotation: Supported
Automatically Adjust Brightness: No
Connection Type: Thunderbolt/DisplayPort

You are drawing twice as many pixels when you drag it onto the 2nd display, if your system specs are only just able to keep up with your timered 25fps on your 1st screen they will have no chance on the 2nd display. Invalidate will only allow refreshes if the system has enough time to do so, if it can’t you won’t see 25fps.

At least try dropping the 2nd display into native resolution for a minute to test if that is the problem.

Have you timed the timer to system.debuglog to see if it is changing frequency? I don’t know how the timer works on the mac but on windows you can saturate the message queue which will seem to slow the firing of timers.

If I’m reading this right, the external display has fewer pixels than the built-in. The external is rendering at 1080p, the internal at 4K. So the external should be faster.

As far as I’m aware, the external 4k display is being used at 200% scaling to it seems as “large” as a 1080p screen but with smoother edges. If the screen was being driven at 1080p “native” you wouldn’t see the two separate lines with different resolutions. But I might be wrong as I’m on mobile so can’t test it for myself.

It’s not telling us anything about the external display. Those stats are all for the internal. The native resolution is 2560x1600, rendering at 3840x2160@2x.

Am I reading the same thing as you?

100% scaling, note the lack of “ui looks like”

U32J59x is a sansung 32" monitor, unless Aaron has some how internalised that monitor, then that is definitely an external screen.

Nope, you’ve got it. My eyes glossed over the U32J59x because it’s a monitor model number, and those look like little more than bashing your face on the keyboard.

1 Like

You’re going to have to look at your drawing code and optimize it for this situation. Part of the trouble is that canvas drawing is done on the main thread along with other things. It is possible in the macOS to use a different thread for drawing, however Xojo doesn’t support concurrency, so enabling this option would probably crash your application.

Below is some suggestions that I’ve made on a different forum. The main principles still apply. Keep it as light as possible, don’t do calculations in the paint event, make sure that pictures are pixel perfect (don’t rely on scaling at draw time). disable anti-alias if you can, avoid calling functions/methods as much as possible, same with loops.

There is a new one, which is to consider adopting the graphics paths with newer versions of Xojo, create these paths outside of the paint event and only paint them in the paint event.

It’s taken me a lot longer to be able to sit down and look at your code.

I’m using 2018r3 for production (as there’s some issues with newer versions of Xojo that I haven’t found the time to resolve yet).

The immediate things I see, that I would do differently.

  • Remove everything that isn’t absolutely necessary to drawing, including the event that show when rendering is started / stopped.
  • Don’t cache the image, draw directly to the canvas. Ensure that the tiles are pixel perfect for the screen. i.e. Don’t get the macOS to scale the images, it looks like if I run this on a Retina machine, the OS has to scale the images to fit the pixel areas.
  • Reduce the number of steps in your paint event, get it as close to a single function as possible, each time it has to call another function or method that adds an overhead.
  • Do all tiles calculations elsewhere, I see there’s a function “computeVisibleTileIndexes” that gets called when the cache is refreshed, do this somewhen else, like setting up the map or when an event happens.
  • Loops have an overhead, when you recalculate which tiles should be onscreen, stuff them into an array, along with their positions, when drawing, loop through a single array and extract the pre-computed locations.
  • Separate the graphics from the tiles, so that tiles which use the same picture/image can use the same picture/image. The OS does some caching, so if you have a 100 tiles that are the same pixels, but different image instances, it has to draw 100 different images. Whereas using the same image, it should be able to gain a bit of performance (if Apple still care about performance). This will also keep your memory down.

I understand that you want to make this as x-plat as possible, but in-order to gain best performance, I would recommend considering using some platform specific hacks.

Hope that this helps in some way. Oh and turn off anti-alias.

Thanks, and Thom McGrath, it is an external Samsung monitor, and yes that is scaled (I use it for larger text, not for for more workspace, because my eyes are not good).

And thank you Sam Rowlands, I had tried already to optimise it in all those ways except scaling, and I guess that is what’s causing the lag. But I’m confused by the advice “Don’t cache the image” (which I notice you also didn’t quote to me but it’s there in what you copied for me) …

I’m going to try making the cached image pre-scaled and see if that helps. I don’t know why I missed this before anyway, because the image only needs to rescale when the window is resized, so it should be cached. (unless there is some wisdom in “Don’t cache the image” but I’m guessing that was something quite specific to your project).

On the Mac at least the canvas contents are already cached by the macOS, if you cache it yourself, it adds overhead, so I would recommend doing as much direct drawing in the canvas as possible. Unless you’re doing so much drawing that caching it yourself is faster, then…

Turns out, it was in fact image scaling. Now I scale the image only when the window is resized, and cache that, then draw that to the canvas. This works as expected, just as fast on the external display as on the built-in.

// previous - this results in slow drawing on the external display
g.DrawPicture( Pic, 0,0, g.width,g.height, scrollX,0, Cached_ScaledPicWidth,Pic.Height )

// new - drawing is as expected
g.DrawPicture( PicScaled, 0,0, g.width,g.height, scrollX_Scaled,0, Cached_ScaledActualWidth,g.height )

Thanks very much for all your help :slightly_smiling_face:

P.S. the “Cached_ScaledPicWidth” and “Cached_ScaledActualWidth” might be a little confusing … I need them because in both cases the source image has been scaled. It took me several hours to implement this change, would have been much better to have realised this at the outset.

2 Likes

Nice solution! On macOS, if you are running at a “weird” scaling ratio (125%, 150%, 175%), I believe the OS renders at 200% and then downsamples. This can result in some extremely large renderbuffers - e.g. a 4K monitor (3840x2160) at a non-integer scaling size could be rendered by macOS at 5K or 6K before downsampling. That’s a lot of pixels. Keeping it 1:1 or 2:1 is going to be much faster.

Right - the answer really depends on those factors: how long does drawing take? how often are you re-drawing (as opposed to just scrolling or refreshing)? are you able to do a 1:1 or 2:1 buffer size? etc.

I did a whole lot of testing a couple of years ago and found that if you’re going to draw pictures, you get the best performance by doing the following.

  • Use window.bitmapForCaching to get a pixel perfect image for the screen (1:1 ratio), or use declares to get the OS to give you a NSSize / NSRect for the screen.
  • Disable anti-aliasing when drawing the pictures.
  • Do not recreate the cache from the Paint event.
  • Use pragmas to prevent other actions during the paint event.

The only complications that comes from this is knowing when the window changes screen, knowing when live resize has finished, for both of these I resorted to using declares and callbacks.

I still miss CGLayers as these were bitmap images on the GPU, there were insanely fast to draw and update. CALayers need a new image each and every time.

Thank you for these helpful tips. I’m sorry I still don’t understand some things …

Window.BitmapForCaching seems like it should be useful, but it is not very well documented, and it seems in any case very poorly named, like a name I might come up with when I’m programming too quickly. Could we come up with a better name for this to describe what it is so I can understand it? Is it a HiDPI picture? (Then how about calling it HiDPIPicture instead?) I honestly cannot understand the reasoning behind “for caching” when I’m being told in the same breath not to cache, and I still don’t quite understand what “recreate the cache in the paint event” even means. I’m sorry to be so dense; thank you for your patience.

Re: Caching. I think the advice depends on the situation. I agree with Sam that macOS tends to do a lot of caching already, so if you keep a copy of your picture (in an attempt to speed things up) you may actually slow things down.

On the other hand, it really depends how long it takes your app to draw.

One way to think of it:

  • if your app can always draw everything in under 1/60th of a second in all situations on all computers, then you can safely do it inside Canvas.Paint with direct drawing operations
  • if not, you should draw to a picture outside your Canvas.Paint operation, and when it’s complete, call Canvas.Invalidate (which will trigger Canvas.Paint). You should consider what your app should do if the Canvas.Paint event fires and the drawing is not yet done (e.g. put up a gray screen with “…rendering…” or similar?)

Basically, when doing Retina correctly, the dpi of the destination context becomes irrelevant. If you wanted a picture that is 128 x 128, you’d call bitmapForCaching and give it 128, 128.

You then draw your stuff into the picture without a care in the world for the technical details of the screen, and at paint time draw your picture.

Under the hood, on 1x screen it creates a 128 x 128, 72 dpi image. On a 2x screen it should create a 128 x 128, 144 dpi image (I say should because there are bugs with older versions of the macOS and Xojo, which means it may not). The macOS will take care of making sure the everything looks appropriate for the screen.

Does this make sense?

Thank you, yes it does make sense. I remember using this also some years ago for a case where a normal Picture was giving blurry results on Retina (I can’t remember what that was exactly). But since then I’ve had no problems using normal Pictures. In this case, I’m not using this function, and the pictures look correct on either display, so as long as that’s the case, I’ll continue not using it.

That said, I’ve added Feedback case 63790 to my “top cases”. Thank you again, Sam Rowlands.

1 Like

Just a note:
In my tests I did a couple of years ago, 1x pictures on a 2x screen were slower than 2x pictures on a 2x screen, even thought the 2x picture had 4x the data, interpolation (to simulate a 2x picture) was slower than actually drawing all the pixels.