WFS / WndProc and OverlayMBS

This is a weird one.

I have a Win32 app which has:

  • a regular window which uses WFS to set up a WndProc to catch some messages (such as WM_POWERBROADCAST)
  • an OverlayMBS window “Overlay1” which serves as a translucent floating display
  • both windows share an Interface called “Layer” which has some common features (hide/move/show etc.)
  • the overlay window is programmatically set so that it moves with the regular window:
Window1.move
  Overlay1.Move(...)

Using earlier versions of REALbasic and Xojo, this all works fine.

Starting with Xojo 2013 R2 or perhaps R3, this app started hanging.

Stepping through the debugger gives something very odd. At some point, when Overlay1.Move is called, the WndProc for Window1 is called. Upon stepping out of the WndProc, the app locks up.

Naturally, I figured that this was my fault for messing with WndProcs, so I removed that code entirely, but the problem remains.

I’m a little fuzzy on how Windows WndProcs and class/subclasses are maintained, but I’m wondering if perhaps there’s something different in Xojo R2 such that Windows subclasses are getting confused by the runtime - or perhaps there’s some conflict between the class names being used by Xojo and MonkeyBread? The fact that both of these items (Window1 and Overlay1) have the same Interface is making me suspicious that this could be the case.

Naturally, in a trivial demo app I’m not seeing the problem.

Any WFS or WndProc gurus out there have any ideas?