Best strategy for REST api usage

General
1

I’d appreciate some advice on the best strategy for writing a REST client. My application talks to a RESTful web api using a variety of api methods, with different parameter sets, different verbs (mostly GET and POST) and different return types (usually JSON, but not in every case).

I’m using the sample REST application as the basis for my application, so I’m creating a different RSocket (Xojo.Net.HTTPSocket) object for each api call, but this is resulting in more such objects than I think is sensible. I can’t quite see how to re-use a Xojo.Net.HTTPSocket, because the PageReceived method that processes the response is so different from one api method to another.

Can someone advise the best strategic approach when (re-)using an HTTPSocket? Am I best off simply creating a new one for each new use?

2

Write your response handlers as methods in a new class named, say APIConsumer. Construct your socket, and give it an instance of the APIConsumer. When a response comes into the socket, have it simply pass that response to the APIConsumer, which will figure out what sort of response it is and process it accordingly.

This approach is really just a way to compartmentalize your response handlers for each API call so you don’t have a giant select case statement or series of if’s in your socket pageReceived event. Makes things much easier to debug / extend in the future.

Also keep in mind that xojo.net.httpsockets are asynchronous, so design all your API calls so that they can be done in an async manner - i.e., don’t expect that you can tell your socket to submit an API call and have the return from that contain the response … the response will show up whenever it’s darn good and ready.

3

[quote=378002:@Kimball Larsen]Write your response handlers as methods in a new class named, say APIConsumer. Construct your socket, and give it an instance of the APIConsumer. When a response comes into the socket, have it simply pass that response to the APIConsumer, which will figure out what sort of response it is and process it accordingly.

This approach is really just a way to compartmentalize your response handlers for each API call so you don’t have a giant select case statement or series of if’s in your socket pageReceived event. Makes things much easier to debug / extend in the future.

Also keep in mind that xojo.net.httpsockets are asynchronous, so design all your API calls so that they can be done in an async manner - i.e., don’t expect that you can tell your socket to submit an API call and have the return from that contain the response … the response will show up whenever it’s darn good and ready.[/quote]

Thanks for the reply. I think that is what I have done, which is good news. I have a class called RESTSocket, which is a subclass of Xojo.Net.HTTPSocket. I am then making a separate RESTSocket object for each asynchronous connection that I am making. I am then passing the Content in the PageReceived event to a method where I build a JSONNode out of the content, and I use that JSONNode object for subsequent processing. I check whether the JSONNode is populated before doing anything that depends on it, which hopefully deals with the asynchronicity.

Question though; is it good practice for me to create all of these RESTsocket objects and not specifically dispose of them? Or are they destroyed when they go out of scope? I am currently generating five of these in the application.

4

Glad you figured it out!

5 sockets is no big deal - unless you are continually constructing 5 new sockets all the time and just letting them go out of scope. I would not recommend doing that. I tend to construct a single socket and do ALL my calls through it. This guarantees that it does not go out of scope (I’ll make it a property in app, or in a class that is a property of app so it does not get destroyed until the app quits).

As far as I know, there is no real downside to re-using the same socket for multiple calls over and over and over… there are probably performance limitations, but I’ve never hit them doing REST api calls to various services.

5

[quote=378002:@Kimball Larsen]Write your response handlers as methods in a new class named, say APIConsumer. Construct your socket, and give it an instance of the APIConsumer. When a response comes into the socket, have it simply pass that response to the APIConsumer, which will figure out what sort of response it is and process it accordingly.

This approach is really just a way to compartmentalize your response handlers for each API call so you don’t have a giant select case statement or series of if’s in your socket pageReceived event. Makes things much easier to debug / extend in the future.

Also keep in mind that xojo.net.httpsockets are asynchronous, so design all your API calls so that they can be done in an async manner - i.e., don’t expect that you can tell your socket to submit an API call and have the return from that contain the response … the response will show up whenever it’s darn good and ready.[/quote]
That is very useful, thanks. Understanding the asynchronous nature of these calls, is there a way to check when the response is complete? I can’t see anything obvious in the docs.
I’m having a problem where clicking a button more than once triggers an error saying another request is in progress (or words to that effect). I could have a pause with a timer, but that feels kludgy and is not guaranteed to work anyway (say, on a slow network).

6

Have a look at httpSocket’s receiveProgress event: http://documentation.xojo.com/index.php/HTTPSocket.ReceiveProgress

7

Thanks, that is useful as ever. I see that this event returns a number of bytes received (in a chunk, not the total) and the total bytes available. When I pick those up I see values for bytesReceived as a number between 13500 and 47649. The totalBytes is always -1. Not sure what to make of these.

I had been thinking that if I had a way to monitor the progress at regular intervals and if the total number of bytes received hadn’t changed since the last interval, the chances, the method has finished returning data. But since bytesReceived is recording chunk sizes, I can’t see how to be reasonably sure that it has finished.

I do get a status code though. But I think that code is sent back at the beginning of the response, so that’s not much use either. Is that right?

And finally, any idea how to interpret that -1 value for totalBytes?

8

Hi Ian,

First off, I pointed you to the incorrect documentation for ReceiveProgress - I accidentally sent you to the classic framework docs, not the new framework docs. You mentioned above that you are using a xojo.net.httpsocket, which is in the new framework. The docs for ReceiveProgress there are here: http://developer.xojo.com/xojo-net-httpsocket$ReceiveProgress

However, they are so similar that the docs are not substantially different. I just wanted to clear that up.

Secondly, the -1 is concerning… but there are at least a few different reasons why it is appearing. My first guess is that the api you are communicating with is not sending the correct headers back in the response to indicate the total size of the response, so the socket has no way of knowing. To verify this, you could use another tool such as Rested (on the mac - I’m sure there are others available on Windows but you’ll have to hit up google) to manually hit the API endpoints and very carefully inspect the traffic (headers and content).

Thirdly: if your ultimate goal is to make sure that you don’t try to re-use a socket that is currently in the process of receiving data, you’ll probably need to build a state machine class that wraps the socket. When the socket makes a request, it enters the “busy” state. When the PageReceived event fires in the socket, it enters the “idle” state, etc.

I have a few other questions that may be helpful as well:
Have you tried checking the PageReceived event to see if it is firing at the conclusion of your download?
How large is the data (in bytes) you are expecting in the responses? 100 bytes? 1000 bytes? 200 Megabytes?
What is the webservice api you are using? Is it something I could try to mock up a quick example for? I may have some time later today I could slap something together.

Good luck!

9

You might want to check out this project:

https://github.com/advancedpricing/Xojo-Rest-Classes

You create a subclass with properties that represent the response, then implement the events to handle it. It makes the process easy and reusable.

10

It could simply be a chunked response. In that case there is no content length sent because the server doesn’t know w ahead of time.

11

[quote=380983:@Kimball Larsen]Hi Ian,

First off, I pointed you to the incorrect documentation for ReceiveProgress - I accidentally sent you to the classic framework docs, not the new framework docs. You mentioned above that you are using a xojo.net.httpsocket, which is in the new framework. The docs for ReceiveProgress there are here: http://developer.xojo.com/xojo-net-httpsocket$ReceiveProgress

However, they are so similar that the docs are not substantially different. I just wanted to clear that up.

Secondly, the -1 is concerning… but there are at least a few different reasons why it is appearing. My first guess is that the api you are communicating with is not sending the correct headers back in the response to indicate the total size of the response, so the socket has no way of knowing. To verify this, you could use another tool such as Rested (on the mac - I’m sure there are others available on Windows but you’ll have to hit up google) to manually hit the API endpoints and very carefully inspect the traffic (headers and content).

Thirdly: if your ultimate goal is to make sure that you don’t try to re-use a socket that is currently in the process of receiving data, you’ll probably need to build a state machine class that wraps the socket. When the socket makes a request, it enters the “busy” state. When the PageReceived event fires in the socket, it enters the “idle” state, etc.

I have a few other questions that may be helpful as well:
Have you tried checking the PageReceived event to see if it is firing at the conclusion of your download?
How large is the data (in bytes) you are expecting in the responses? 100 bytes? 1000 bytes? 200 Megabytes?
What is the webservice api you are using? Is it something I could try to mock up a quick example for? I may have some time later today I could slap something together.

Good luck![/quote]
Thanks. The state machine idea feels right. I’ll think a bit more about how to do this.

I’m using the PageReceived event to grab the data that comes back.

Size of data is about 50K, so we are not talking about massive data packages at all!

The webservice is a private one for one of my clients I’m afraid, so it’s not feasible for me to point you to the details. Thank you for the offer anyway; that is very generous.

Regarding the -1 thing, I use Paw on my Mac, so I will analyse the data coming back from that.

In case it’s of interest, here is a screenshot from an example I put together today. I hit the “Get projects for server” button and the result comes back in less than a second. I used the ReceiveProgress event to add rows to the two list boxes. I always get the 200 status code, and the textarea has the json package that constitutes the result.

Thanks for your continued help with this - much appreciated.

12

Could be. I think I need to do more homework on this…

13

[quote=380990:@Kem Tekinay]You might want to check out this project:

https://github.com/advancedpricing/Xojo-Rest-Classes

You create a subclass with properties that represent the response, then implement the events to handle it. It makes the process easy and reusable.[/quote]
Thank you. I will take a look at this. I have been basing everything on the example project in the Xojo kit, and it looks like this is slightly different, so it will be helpful to check it out.

14

Quick tip, if you need to format the JSON to understand how it is structured you can use https://jsonformatter.curiousconcept.com