Wait until Xojo.Net.HTTPSocket finish

Getting Started
1

This is the first time with asynchronous requests and my mind can’t handle that yet.

I want to connect to a website, request some JSON code and process the results. They limit the number of Items per request to 5 and I need to get 14, and 10 requests per minute. Having:

MySocket.Send("GET", url1) MySocket.Send("GET", url2) MySocket.Send("GET", url3)
display an error because a request is already in progress when the second request is made.

My not-so-elegant solution:

  • make the user click a button to get url1,
  • disable the button,
  • change the caption to URL2,
  • MySocket.Send(“GET”, url1),
  • at PageReceived last command enable the button
  • repeat until caption=“DONE” and keep it disable

I’m sure there are other (and better) ways to do this. My goal is that the user only click the button once. I’m reading about threads and timers (those are new to me too).

2

The simple solution would be to create an array of the URLs that you need to request.

The first click of your button would pop a URL from that array and use your socket to send a GET request (you could also disable the button here).
Then, in the the PageReceived event, process the JSON that was just returned and then pop the next URL off of your array and send that request.

Repeat that process until your array is empty (and enable the button once again if you want to allow this process to happen again).

3

You’ll need to exit the page received event before making the next call. I use xojo.core.timer.calllater(0, addressof ) which makes the request in the next event loop.

4

Another option, that would take better advantage of the asynchronous sockets, would be to create multiple instances of MySocket and perform all of the requests simultaneously.

This is slightly more complicated, since you would need to add each instance of MySocket to an array that is a property of a module or window (so that it doesn’t fall out of scope) and then remove them when they are finished processing the request.

Something like this in your button:

[code]dim s as MySocket

for each url as string in getRequests
s = new MySocket
s.Get(url)
socketArray.Append s
next[/code]

Then you would need to remove the socket instance from that array when it is done with the request:

Public Sub RemoveReference(sock as MySocket) for i as integer = socketArray.Ubound downto 0 if socketArray(i) = sock then socketArray.Remove(i) end next End Sub

So in the PageReceived event you would do your processing then call:

RemoveReference(self)
5

Depending on the number of requests that may very well get you blocked from the server if it thinks you’re a DDoS attack. I would recommend a queue over this approach.

6

Very true.

Throttling the requests based on the service’s requests/minute limit does get more complicated than necessary, so the queue is a better approach.

7

[quote=367650:@Jared Feder]The simple solution would be to create an array of the URLs that you need to request.

The first click of your button would pop a URL from that array and use your socket to send a GET request (you could also disable the button here).
Then, in the the PageReceived event, process the JSON that was just returned and then pop the next URL off of your array and send that request.

Repeat that process until your array is empty (and enable the button once again if you want to allow this process to happen again).[/quote]

Thank you Jared.

I’m doing most of what you say, a) first click, b) GET request, c) process the JSON, but I don’t know how to send the next request without the user clicking the button again.

Thank you Wayne.

This works:

Xojo.Core.Timer.CallLater(1000, AddressOf GetURL1) Xojo.Core.Timer.CallLater(5000, AddressOf GetURL2) Xojo.Core.Timer.CallLater(9000, AddressOf GetURL3)

I guess I have to figure out timeouts or other errors. If one of the requests stay active and the second fires up.

@ Jared, thanks again. I’ll take a look at the code and do some tests.

@ Tim, thank you. I only need to do 3-4 requests. They allow 10 per minute and 2000 per day. I have to provide a token and they can block and let you wait some time until you can do your next request, it could be several minutes or until the next day.

8

[quote=367671:@Alberto De Poo]Thank you Wayne.

This works:

Xojo.Core.Timer.CallLater(1000, AddressOf GetURL1)
Xojo.Core.Timer.CallLater(5000, AddressOf GetURL2)
Xojo.Core.Timer.CallLater(9000, AddressOf GetURL3)[/quote]

That can go wrong if the service takes an abnormally long time to return a result.

What Wayne was referring to was to create a method called GetNextURL that would look something like:

Public Sub GetNextURL() if urlArray.Ubound = -1 then MsgBox("We're done with all of the requests") else MySocket.Get(urlArray.Pop) end End Sub

So in the button you would put:

[code]redim urlArray(-1) // this is a property of your window

urlArray.Append(“https://my/first/url”)
urlArray.Append(“https://my/second/url”)
urlArray.Append(“https://my/third/url”)

GetNextURL[/code]

Then in the PageReceived event you would process the returned JSON and then call:

Xojo.Core.Timer.CallLater(10,WeakAddressOf GetNextURL)

That will pop the next url off the the array and send a new GET request via the same socket.

9

Thank you Jared, that makes more sense.

I learned several things today.

10

From my experience, no need to worry how long it takes even 0 works fine to have the page received event finish
Xojo.Core.Timer.CallLater(0, AddressOf GetURL1)