HTTPSecureSocket.get and JSON data

Hi all,

I am new to XOJO but have tried RB some time ago. I am trying to parse a Guild Wars 2 API website that produces data in JSON format, but I am running into the issue of the “Download Complete” not firing for some reason.
On Window.Open event, I have placed:

Dim myhttp as New HTTPSecureSocket
myhttp.Secure=true
myhttp.SetRequestContent("","application/json; charset=utf-8")
myhttp.Get("https://api.guildwars2.com/v1/maps.json")

When I inspect the myhttp in debugger, it shows that data is loaded in 16k chunks. For some reason, no matter how long I wait PageReceived does not fire at all. The totalbytes variable indicates there are 147k bytes. Should I be using something else to connect and receive the data from the site?

Many thanks for your advice,

Patrick

I would make the myhttp object a property of the window as the myhttp.get is asynchronous & the object may go out of scope before all the data is received. Also I find with TCPSockets that I need to add a timer to poll the socket during receive which may or may not be necessary here.

How are you handling the PageReceived event? In your posted code PageReceived has no event handler.

There’s another possibility. Our http sockets currently use the 1.0 protocol. If the server is expecting 1.1 and can’t handle a 1.0 request, it may not know to close the socket when its done. Only the server can do this though.

Thank you very much for suggestions. Wayne, I have since declared the myhttp object as a property of the window as you suggested, but unfortunately the same results. Andrew, in the PageReceived, I had the following to find out what the length was of the resulting buffer.
I guess I could be checking if the length of bytes received and bytes to download match and continue to wait? I am not sure what would work.

Msgbox str(len(content))

Greg, I am not sure about the server http socket protocol. Is there any way I could check what version is being used? Alternatively, would there be a workaround for version 1.0, if this is indeed the case? What I can say though, is that the server sends httpStatus 200 as soon as the first 16k of data is received by the socket.

That’s not quite what I meant to ask for. What I mean is how are you associating this code with your socket?

Your first code snippet showed that you are instantiating a generic HTTPSecureSocket as a variable with scope local to the Window.Open event:

What I was asking was which method are you using to associate your handler code with the HTTPSecureSocket instance?

Hi Andrew,

thanks for the post. I am sorry, I am not really sure what you mean.

Patrick

Where is this code located and how are you telling the socket to call it?

Msgbox str(len(content))

Sorry, now I understand. This particular piece of code is in the event PageReceived of the myhttp socket (property of the window).
I initially had declared the socket in the Window.Open but Wayne’s suggestion made much more sense, to attach the socket to the window.
I hope this makes sense.

So currently this is what I have:
–Window.Open event

myhttp.Secure=true
myhttp.SetRequestContent("","application/json; charset=utf-8")
myhttp.Get("https://api.guildwars2.com/v1/maps.json")

–myhttp.PageReceived event

Msgbox str(len(content))

–myhttp.DownloadComplete

Msgbox "Done download"

Ah, I see now.

HTTPSecureSocket.Get has four versions. Two versions trigger events and two versions don’t. The version that triggers the DownloadComplete event is HTTPSecureSocket.Get(URL As String, File As FolderItem). The version that triggers the PageReceived event is HTTPSecureSocket.Get(URL As String), which is the one you’re using.

Given this, what would the best way be to make sure that the entire file is downloaded, instead of the first 16k of data?

When I use an HTTPSecureSocket to retrieve https://api.guildwars2.com/v1/maps.json I get the whole 140K+ of the data in the PageReceived event. The only place where I see only 16K is in the ReceiveProgress event which fires approximately every 16K. I don’t know why it would stop after 16K or how to prevent it.

Thanks Andrew. I am not sure why for some reason my code is not working. Would you mind sharing what you did?

Sure, here’s the project file that downloads the whole file for me:

Thank you very much Andrew, this works! I am not sure why my script was not working…