Modal dialog dismissed, debugger stuck

I just experienced a weird Xojo behaviour I’ve never seen before and thought I should report it here instead of Feedback because I can’t reproduce it.

When quitting my app, the user is prompted with a modal dialog to Save, Don’t Save, or Cancel. The modal dialog is shown using .ShowModalWithin Normally the user selects the desired option, and as long as it’s not Cancel, then the dialog closes, the main window closes, and the app quits.

This time, I selected Quit (without saving) and the dialog went away … but the app did not quit. Instead it froze there, with the main window still open and the app menubar highlighted, as it does when it’s quitting, but … it was obviously stuck. There was no beach ball, so I paused the debugger. It was stuck on the line of code dialog.ShowModalWithin(w) So it was still waiting for the dialog to be dismissed (closed). But the dialog was closed. No way out of that one except to force quit. Anyone ever see this happen before?

I have seen this happen in circumstances where the Close event refers to some attribute of the window it is closing, thus reinstantiating it again…

That makes sense. But this is a dialog. It has no close event. Here is the global method which instantiates the dialog. I use it all the time for warning dialogs. I just discovered that I had added a comment long ago to another app about this very same behaviour happening, and now I remember it was when the Cocoa framework was first being introduced. For a while I had to stop using ShowModalWithin because it would always lead to this “coma”. Well, it’s happening again in the latest Xojo.

Public Function WarningBox(actionCaption as string, alternateCaption as string, cancelCaption as string, heading as string, msg as string, w as window = Nil) as string
msg = DefineEncoding( msg, Encodings.UTF8 )

Dim d as New MessageDialog
Dim b as MessageDialogButton
d.icon = MessageDialog.GraphicNote
d.ActionButton.Caption = actionCaption
if cancelCaption <> “” then
d.CancelButton.Visible = True
d.CancelButton.Caption = cancelCaption
end if
if alternateCaption <> “” then
d.AlternateActionButton.Visible = True
d.AlternateActionButton.Caption = alternateCaption
end if
d.Message = heading
d.Explanation = msg

if w <> Nil then
b = d.ShowModalWithin(w)
else
b = d.ShowModal
end if

return b.Caption
End Function

if the app is quitting, it is trying to close the windows… so if you refer to a window that is attempting to be closed (ie. “W”)

If any of this is in the CLOSE event of the APP or any WINDOW, try moving it to CANCELCLOSE instead, or set W=nil

Dave, thanks for trying to help, but that isn’t the problem.

The dialog is shown correctly. It is called from the main Window’s CancelClose event. That’s all correct.

The problem has to do with closing the dialog after it is shown using ShowModalWithin. When it is dismissed by the user, it closes visually, but somehow does not close inside Xojo’s execution. Xojo does not advance the code forward.

I’ve found a clue. This only happens if I have at any point opened a window of type “Sheet” in the main window. A Sheet Window can be shown within the main window using only Show, not ShowWithin. This is a convenience on Mac. But it is creating this problem. If I don’t happen to have opened any sheet windows during the session, then the dialog doesn’t hang, and the app quits as expected.

Egads, it’s all coming back to me now. I ran into this same problem years ago and solved it somehow here on the forum. I don’t know if it was this same forum or the old one. Rats. I don’t know what the solution was, but I know I found it on the forum. I’ll do some searching…

Nope, false alarm. That was https://forum.xojo.com/27498-returning-true-in-window-cancelclose-not-working a different problem.

Found it: https://forum.xojo.com/26220-dismissing-dialog-using-showmodalwithin-does-not-return-to-code

However, the previous solution is not working. The modal dialog has been dismissed by the user. It closes, visually. Xojo simply does not return to code execution. It sits there on this line:

b = d.ShowModalWithin(w)

I haven’t mentioned that when this happens, the window’s close button stays in the darkened “clicked” state, the user can click this close button again, and the dialog will reappear, can again be dismissed, and Xojo continues to sit on this line. That can be repeated ad infinitum. The debugger stack trace shows a series of CancelClose > SaveCheck > WarningBox events. In the WarningBox method (where the dialog is shown) the button b is shown as Nil every time.

An obvious workaround is simply not to use .ShowModalWithin, and instead use .ShowModal, but since I wanted sheet window behaviour, that was not acceptable.

The only solution was to use MBS. I subclassed NSAlertMBS and made it behave like Xojo’s sheet dialog, which required a lot of fiddling.

Earlier versions of this same project do not show this behaviour. I still have no idea what causes it. It is not to do with the Activate event as described in the other post about the same problem. I looked for circular window references and found none.

Since sheet windows are Mac-only, I am now branching the WarningBox code so that Windows uses the method above (without the showWithin option since sheets don’t exist on Windows), and for Mac it’s now a bit complicated under the hood using MBS, but it works.