Most efficient way to queue web requests

I’m building an application that will receive requests from multiple apps using WebApplication.HandleSpecialURL. These events need to be processed one at a time in the order they are received, making sure that one process is completed before the next one starts.

Does anyone have any experience doing this or suggestions of how to do this efficiently, so that the waiting requests use a minimal amount of CPU?

Thanks!
Bart

If I remember correctly HandleSpecialURL blocks itself. So if three clients request a special URL they are processed serially in the order that they are received. Pending HandleSpecialURL events shouldn’t take any CPU time while waiting for their turn. (A quick test seems to confirm this, but someone from Xojo would need to confirm if you have to be 100% sure.)

If the tasks complete quickly I don’t think you have to worry about it. They should complete in order.

If the tasks take a long time (i.e. a waiting client might potentially time out) and you do not have to return the results at the end of HandleSpecialURL, then I would:

  • Collect the information needed for the task.
  • Store that information in the properties of an abstract object which represents the task.
  • Add that object to a global array.
  • Return something to the client to indicate “received” and exit the HandleSpecialURL event.
  • To handle the actual processing I would have a Threaded loop periodically check the array; get/remove the next task; and process it. Thread.Run and HandleSpecialURL should not block each other. (Always keep in mind though that Xojo cooperatively schedules things so they won’t run at the same time on separate cores. One simply doesn’t have to wait for the other to complete to get some of the shared time.)

HandleURL and HandleSpecialURL are threaded just like all other requests coming into the web framework. This means that they may yield to another thread on a loop boundary or if you call App.DoEvents or App.CurrentThread.Sleep. From the Thread docs:

[quote]When the Thread Scheduler decides to stop execution of the current thread and allow another thread to run, it is called a context switch. The amount of time a thread runs is called the time slice for the thread.

Threads can yield time to other threads and other applications each time they execute a looping construct such as For, While, and Do. However, a thread does not necessarily yield time at every loop boundary even though it has an opportunity to do so. A thread actually yields to another thread when the Thread Scheduler decides that its timeslice has expired. Context switches are expensive, so the Thread Scheduler always tries to avoid them.

You can also force context switches by calling Application.YieldToNextThread or by calling Application.SleepCurrentThread.[/quote]
What this means is that if your handler code takes more time for certain requests, they may not finish in the order in which they were received.

The way I handle this in my own code (similar to Daniel’s solution):
0. Create a property on the App class of type Timer (ParseTimer as Timer)
0. Create a class which represents the data that you need to deal with (you could just use a string if your data is json data or something line that)
0. Create an array property on the App class of the type defined in the last step (MyData() as MyDataClass)
0. Create a method called ParseTimerAction(t as Timer) which pulls the first element off the MyData array (MyData(0)), Removes the element from the Array ( MyData.Remove 0 ) and parses the data.
0. When data comes in create an instance of your data class and add it to the end of the data array: MyData.Append newData
0. In that same method, make sure the Timer is running, configured and its action event will fire the ParseTimerAction event

[code]if ParseTimer = Nil then
// Set up the timer if it’s not already running
ParseTimer = New Timer
// The value of the Period property depends on how fast you need your data parsed.
// Lower values mean the data will be processed more often, but will also use more CPU.
ParseTimer.Period = 10
ParseTimer.Mode = 2
AddHandler ParseTimer.Action, AddressOf ParseTimerAction
End If

// Resetting the timer means that processing will be deferred until there’s a break in incoming data.
ParseTimer.Reset[/code]
7. In the ParseTimerAction method, I suggest wrapping the code in a While loop like this:

// Stop the timer while we're processing data (so we don't end up with more than one)
ParseTimer.Mode = 0 

// Loop through the array and parse the data in the order in which they were received
While Ubound(MyData)>-1
  Dim CurrentData as Data = MyData(0)
  MyData.Remove(0)
  // Parse CurrentData here
Wend

// Let the timer continue when we're done
ParseTimer.Mode = 2 
ParseTimer.Reset
1 Like

If the incoming request doesn’t need the results, I’d …

  1. Persist the incoming data in a database table.
  2. Acknowledge successful receipt to the request.

Use a Timer to check for unprocessed data in the table…

  1. Mark first available record with timestamp as being processed (so other instances of App do not process it unless timestamp has “expired” without completion).
  2. Process data.
  3. Mark database record as processed (maybe delete it, or use a timestamp to mark it as processed and at some point delete old records during empty timer actions when processed timestamp is over X days ago).

This way the queue persists through events that bring down the App (planned hardware/software updates, crashes, power interruptions etc) and allows for processing the queue in separate processes to the web app itself (e.g. multiple console apps).

I was about to ask you for a clarification on this point when I realized where my test went wrong.

In the test the first call to HandleSpecialURL would block for 30s with a loop that called CurrentThread.Sleep. Additional incoming calls were supposed to complete immediately and .Print a confirmation. I was opening new tabs in FireFox to try it out. They would all block, then rapidly complete in order when the first cleared.

FireFox sees the same URL and waits for the first tab to complete. It doesn’t copy those results, the confirmation text was correct for each tab. But it waits on the first one. Add in some parameters (i.e. http://127.0.0.1:8080/special/?id=1) and the subsequent calls complete immediately. It’s a different URL to FireFox.

I think I did a similar test before which led me to believe that HandleSpecialURL would block itself and complete serially per IP address. My apologies to Bart for any confusion, and thanks to Greg for clearing this up.

Having said that…

[quote]// Stop the timer while we’re processing data (so we don’t end up with more than one)
ParseTimer.Mode = 0 [/quote]

This caught my eye. Will Timer.Action fire while a previous Timer.Action (same instance) is in flight? Is this only in web or console? I seem to recall…going all the way back to the old forums…confusion over this. My lingering impression is that in desktop apps a particular Timer.Action will not fire again until the last event has completed. Obviously other Timer instances can fire their Action events.

It’s habit. I sometimes use a subclass of Thread as a timer.

Thanks Daniel & Greg for taking the time to help me out!

Bart

Thanks Ian!

Bart