Updating progress indicators during heavy processing

[Mark_Ogier]

Hi there

[quote]Here’s how I’d handle it:

Create a Task subclass designed to do this export. It will contain all the information is needs to complete its task. That can include an array of files to export.
When the user clicks export, store this information in the Task’s properties, disable the interface, then run the Task.
React to the Task’s events to update your progress bar. When it tells you it’s done, re-enable the interface.
At no time do you create a loop to monitor the Task, you let it feed you the information instead.

If you don’t want to handle disabling the interface, move this function to a dialog window and have it do the processing in the way I described. When the Task is finished, it will close the dialog.[/quote]

Thanks so much to everyone for their help with this. I feel I am slowly getting the picture. As Kem said, my experience is with pretty linear programming requirements. Never really used threads much, not even to update progress bars!

Can I just clarify a couple of points pls?

1. What is being suggested is pretty much going to put the entire code-base for my export process in the Run event of the Task. Feels wrong to me to not be able to make the code more modular and break it up more. Given I am in a Method of the Task Class (Run), what can I safely access from a ‘Scope’ perspective ? (I read it is limited). Can I access Globals? Can I access Form level Methods & Properties? Or does everything have to be self-contained in the Run method of the task?

2. Reacting to the Tasks events. So are you saying that I create an event in the Task sub-class that gets raised when the export has finished? And in that event put the code to re-enable the UI? More new territory Not created my own events before…

3. So working in this way presumably means I must not have any code that linearly follows the call to Task.Run, otherwise it will get run when the Thread surrenders time back to the main app? So basically, in my Go button, one line, Task.Run (if we exclude all the property setting up stuff to initialise values in the Task). After that nothing.

4. The updating of the UI (the point of all this headache!) is still achieved in the Task using the same UpdateUI method that is in the Xojo example Task code?

Sorry to be a pain about this, as you say, it is a relatively new way of thinking for me. I am used to events in a single thread sense, but the interaction between the main app and a separate thread, with events between the two is a bit of a challenge right now. But thank you for hanging in with me

Regards
Mark

[Kem_Tekinay]

You’ve got it all right, except the Run event in the task can call methods, function, and access properties. If the Task is part of the Window, it can access any part of the window that is not part of the UI and any global properties, etc. (Note that any methods or functions it calls will run in the same Thread exactly as if you had coded them inline.)

The Task example will raise an event for you when it’s done, you just have to react to it.

(BTW, not sure if it’s clear but you can drag a Task (or Thread) directly into a Window and it will appear as just another control.)

As for my project, download it, open the example project, then copy and paste the Looper_MTC class from there into your own project.

[Mark_Ogier]

OK, thanks, I am on it! More code re-jigging! I WILL get this
Thanks for all the help, I will report back soon…

[John_McKernon]

@453029:@Kem Tekinay

As an aside, I’ve create a class I call “Looper” that uses a Timer in the Main Thread to emulate processing in a Thread. It’s designed around a loop and its main advantage over the Task example is that it still provides full access to the UI. Now I just have to decide how best to distribute this.

After all these years, I finally understand what you (and others) have meant about putting what is otherwise in a loop into a thread. This is so clean and elegant, I can’t wait to give it a try, thank you for being so generous with your time and talent!

[Kem_Tekinay]

Oh don’t, stop.

Hmm, seems I added an unnecessary comma there…

[Mark_Ogier]

Just wanted to report back on my results after so many questions and so much help from everyone here.

So After a bit (a lot actually) of code re-jigging, I now have a very nice responsive UI with status updates showing very smoothly. Thank you so much to everyone that pitched in and help.

For the sake of anyone else who may be contemplating the same challenge at some point in the future, here is an overview of what I did.

First off I used the Task class provided in the Xojo Desktop examples. I will work at using Kem’s looper class at some point in the future but as I had already started out by using the Task solution I stuck with that for now.

I then sub-classed the example Task class. Real easy to do in Xojo

I then dropped my sub-classed task onto the form I was using that needed the UI updates & progress indicators. It then just shows as a control on your form in the code explorer on the left of the main Xojo window.

I then added Properties and Methods to the sub-classed Task so that I could pass in all the values I needed to use during the export process and subsequent UI updates. Important to note that you cant access the value of any UI controls on the form from within the class. I had values in labels that I wanted to refer to but could not so I had to pass them in using the properties of the sub-classed task before calling the Run method of the Task

Then all the heavy-lifting code had to be placed in the Run method of the Task. I tried having it split up into methods in various places but Scope became a bit of a headache so it now all just lives in the Run method. There are many methods in the form on which the task sits, and all those are accessible by the task no problem, just prefix the method name with Self or the Form name.

I then just make calls to the ProcessUI method in the Task class, passing in Key/value pairs which are then used to update the UI. This method (ProcessUI) is the only place that the UI can be accessed in the Task class due to it running in a separate thread. But it works just great.

So thanks to all again.

After 25 years programming with Microsoft products I can honestly say that all in that time, the level of community support never came anywhere near what is given here by the Xojo users. Amazon!

Regards
Mark

[Kem_Tekinay]

Great news.

FYI, you can put methods in a subclass and access them from the Window using the Me keyword.

When a control is in a Window, Self means “something belonging to the window”, whereas Me means “something belonging to this control”. Outside of a window, Me and Self are interchangeable.

// Within a window...
Self.MyWindowMethod
Me.MyClassOrSubclassMethod

Ditto for properties.

[J_H_Timmerman]

An interesting simple possibly usefull alternative I recently learned about in the Delphi user forum is an adaptation of the DoEvents- aka ProcessMessages- function, to make it better behaving.
A proposal by Peter Below, as shown in the following article:

https://community.idera.com/developer-tools/programming-languages/f/delphi-language/68847/application-processmessages-behavior/86709#86709

His adapted method, ProcessPaintrequests, is defined in Pascal as follows:

procedure ProcessPaintrequests; var Msg: TMsg; begin while PeekMessage(Msg, 0, WM_PAINT, WM_PAINT, PM_REMOVE) do DispatchMessage(Msg); while PeekMessage(Msg, 0, WM_NULL, WM_NULL, PM_REMOVE) do DispatchMessage(Msg); end;

So only messages related to Paint-events are handled.

The (hexadecimal) constants used are

WM_Paint = 0x000f
WM_NULL = 0x0000
PM_Remove = 0x0001

(see the full list of WM-constants at: https://wiki.winehq.org/List_Of_Windows_Messages).

I tested this procedure in a simple project with a time-consuming loop., where a Progressbar is
updated during the process and the numeric output of the long process is shown in a textfield.

Interestingly, in Pascal (without or with ProcessPaintrequests) the Progressbar was updated
correctly, but the textfield only when using ProcessPaintrequests.

In a similar XoJo-project I saw the same behaviour of the Progressbar and textfield, but I
couldn’t yet test the effect of ProcessPaintrequests, as I didn’t have the correct definitions
in Basic of the Msg-structure.

Perhaps worth a try …

[Michel_Bujardet]

Application DoEvents would indeed enable the UI to update while a tight loop takes place. But it also may have unwanted side effects.

As outlined in the FB at http://documentation.xojo.com/api/deprecated/application.html#application-doevents :

Using DoEvents in a GUI application will likely cause instability.

DoEvents in Xojo are not the same as in Delphi. They are required in Console applications, but strongly discouraged in Desktop apps.

[Mark_Ogier]

Kem Wrote

FYI, you can put methods in a subclass and access them from the Window using the Me keyword.

Quick question/observation.
When I sub-class a class such as the Task class, and add properties/methods to the sub-class, If I then add the new sub-class to my form, the resulting object in the explorer does not show any of the properties/methods I added to the sub-class. They are available from code in the code of the window it was added to but can only be seen in the explorer in my original sub-class.

Is there a reason for this? If I add a property to my sub-class I would expect it to be visible in the explorer for any objects I create from it. And as the properties etc are available in code I would have thought they would show up in the explorer.

It isn’t a big deal but it is a bit of a pain having to keep referring back down to the original sub-class to remind myself of the available properties etc.

Just curious.
Cheers
Mark

[Kem_Tekinay]

You’re right, it doesn’t. You can define which properties show up in the Inspector (right-click the subclass to get to that option), but otherwise rely on autocomplete to show you after the Me keyword.

[J_H_Timmerman]

@Michel Bujardet

Peter Below’s function is NOT using ProcessMessages or DoEvents !
That’s the clue.

It dispatches only a subset of all the possible messages, only the relevant ones for updating the UI.
So it is unlikely that it will cause instability-issues like DoEvents and the like.

I have translated the Pascal-definition of ProcessPaintRequests to XoJo.

For simplicity, I used the definitions of Eugene Dakin’s book on Windows-declares.
However, the definition of the Msg -structure was incomplete; it missed the last field
(“private”, DWORD = UINT32).

Tomorrow I’ll show the function, so one is free to test whether it is usable !

[Michel_Bujardet]

[quote=453597:@J H Timmerman]@Michel Bujardet

Peter Below’s function is NOT using ProcessMessages or DoEvents !
That’s the clue. [/quote]

Oh, so it is a Windows declare. I did not understand - sorry.

Since the question was posted in a general channel, once can imagine the OP needs a cross platform solution, though.

But still, looking forward for the actual code

[Robert_Birge]

Or you can add “app.doevents” to update your progress bar or text field.

My suggestion will generate all sorts of comments on why you shouldn’t do this, and can cause a crash. And while it can cause a crash, it usually does not. The main negative is that it slows things down so you should only call it every two seconds or so.

[Kem_Tekinay]

“Or you could drive this car. Sure, it will occasionally explode and kill you and your family, but the main negative is that the transmission sticks a little.”

[Thom_McGrath]

[quote=453614:@Robert Birge]Or you can add “app.doevents” to update your progress bar or text field.

My suggestion will generate all sorts of comments on why you shouldn’t do this, and can cause a crash. And while it can cause a crash, it usually does not. The main negative is that it slows things down so you should only call it every two seconds or so.[/quote]
Here we go again… I should resist, but education is important.

In Xojo, it doesn’t matter if you’re doing everything on the main thread or multiple threads, only one line of code can execute at a time. This is an important fact to keep in mind. The exception to this rule is plugin code that yields to other threads.

When your app starts, it runs a loop until the app stops. Imagine it looks something like

RaiseEvent Open While True App.DoEvents Wend RaiseEvent Close

DoEvents literally runs one iteration of the event loop. Again, this is psuedo code, but it would look something like

Try DetectInputEvents() PollTimers() PollSockets() For Each Thread In CurrentThreads Thread.ContinueUntilYield() Next Catch Err As RuntimeException RaiseEvent UnhandledException(Err) End Try

Calling App.DoEvents literally does all this again! A timer with a very short period could be triggered by a nested DoEvents call. If DoEvents happened as a result of that timer, boom, you have a re-entrancy problem. And then come the stack overflow exceptions.

What is DoEvents good for? Console apps. They don’t have an event loop, so DoEvents allows you to run your own. In any other scenario, the method is dangerous.

The only reason DoEvents is not illegal in desktop apps is the IDE uses it during the loading window, and HTTPSocket uses it for async requests. So yeah… don’t use async HTTPSocket either. And the loading window needs refactoring.

There is never a reason to nest your event loops. So don’t do it.

[J_H_Timmerman]

To use the function ProcessPaintRequests (by Peter Below) , only for the Window-platform, it is required:

First you have declare two structures: PointMsg and tagMsg.

This must be done in the IDE and the structures must be added to the project (use the “+”-button and from the list
choose structure). You give the new structure a name and add its fields by using the “+”-option.

PointMsg
   x AS INT32
   y AS INT32

tagMsg
   hwnd     AS  INTEGER
   message  AS  UINTEGER
   wParam   AS  UINTEGER
   lParam   AS  INTEGER
   pt       AS  PointMsg
   lPrivate  AS  UINT32

Further one defines three constants in a similar way in the IDE:

  PM_REMOVE  = &h0001

  WM_NULL    = &h0000 
 
  WM_PAINT   =  &h000F

For the API-Function “PeekMessage” there exist two variants: PeekMessageW and PeekMessageA.

In modern Window-versions it seems that PeekMessageW is used, but if necessary one can test if
it is available by using System.IsFunctionAvailabe(“PeekMessageW”, “User32”).

Otherwise PeekMessageA should be substituted in the code below.
The same is true for the variants of the function “DispatchMessage” (W or A).

The code for ProcessPaintRequests is simple:

[code]DIM Msg AS tagMsg

Soft Declare Function PeekMessageW Lib “User32” (ByRef lpMsg as tagMSG, lhWnd as Integer, wMsgFilterMin as UInt32, wMsgFilterMax as UInt32, wRemoveMsg as UInt32) as Int32

Soft Declare Function DispatchMessageW Lib “User32.dll” (ByRef lpMsg as tagMSG) as Int32

WHILE PeekMessageW(Msg, 0, WM_PAINT, WM_PAINT, PM_REMOVE) > 0
CALL DispatchMessageW(Msg)
WEND
WHILE PeekMessageW(Msg, 0, WM_NULL, WM_NULL, PM_REMOVE) > 0
CALL DispatchMessageW(Msg)
WEND
[/code]

[Kem_Tekinay]

Things you should try before using App.DoEvents in anything other than a console app:

1. Drop a brick on your bare foot.
2. Stab yourself in the leg with a sharp pencil.
3. Listen to political speeches nonstop for 20 hours.
4. Ignore anyone who tells you otherwise.

Thom explained why above.

[brian_franco]

[quote=453655:@Kem Tekinay]Things you should try before using App.DoEvents in anything other than a console app:

1. Drop a brick on your bare foot.
2. Stab yourself in the leg with a sharp pencil.
3. Listen to political speeches nonstop for 20 hours.
4. Ignore anyone who tells you otherwise.

Thom explained why above.[/quote]

5. Hang out with me for more then an hour

[Kem_Tekinay]

Wow, way to take it too far, @brian franco …