I’m working on refactoring some of my networking code away from the old HTTPSecureSocket to the newer xojo.net.httpsocket class. I need to be able to inspect the response headers, but I note that in the xojo.net.httpsocket class we only have the headersReceived() event, which does not show which headers were received - just that they arrive. I also see that there is a responseHeader(name as Text) method which can be used to look up a specific header - but that’s not real useful if I don’t know which headers the server returned in the first place.
Short of trying every response header defined in the http/1.1 spec, is there a way to get the xojo.net.httpsocket to just spit out a list of all the response headers it received?
Unfortunately, no, the endpoints I’m consuming do not document their headers, and the headers do change based on the disposition of the response. For instance, if I make a request that the API decides is invalid, it may include a warning header in the response, but won’t include that header on valid requests.
To be more specific: this webservice is supposed to only ever send back valid JSON responses in the response body… but if there is an issue with the processing of my request that causes their webserver to barf, it just pukes out a generic nginx error message.
In the past this has caused all sorts of problems for my app which was blithely expecting well-formatted JSON responses, but gets some HTML error message instead.
So, I started to inspect response headers as a matter of course when processing a response before I tried to parse whatever content I received. Sometimes I found telltale signs in the headers that the response was not going to be parseable. Sometimes I received no response headers, which would also indicate I should not try to process the response as JSON. The point is, the old HTTPSecureSocket’s pageReceived event included the headers as a variable in the event, so I could inspect them with my business logic to see how I should deal with parsing the actual content.
The new httpSocket has removed these headers from the pageReceived event, so I’m trying to find them another way, to avoid barfing while processing a potentially bogus response.
So, I would think, if the header is not there the response value would be empty? I’m assuming here that this functions like Dictionary.Lookup(Key, Default) except the default is always empty. You shouldn’t need all HTTP 1.1 headers, just the ones you’re looking for each time (a reusable dictionary or class for this might be a good approach).
I tend to abstract communications into their own class. The class handles communication and parsing, and then I use the class like a data object. Kind of like ActiveRecord for APIs.
I dislike the new framework when dealing with API responses, so I turn the xojo.core.memory blocks back into strings to work with. Once in the classic framework I try to create a json item from the response. Try Catch JSONException, if there’s an exception there was an error. You can’t always handle the response (especially if the server failed in some way), so leave yourself a way to handle the errors you discover. Personally, I check to see if it’s an error I can handle properly, if not I raise an exception and catch it (thread safe, msgboxing the errors from the class will cause UI exceptions).
If there is a valid json item, I then parse that for expected messaging. In a recent project, I designed a base class that parses for the basic problems and errors, and then I subclass that for each specific endpoint. This way each class is specialized for it’s endpoint, but shares the same core error handling.
That’s about all I’ve got for generic Web API communication advice. More than happy to help where I can, reach out to me by email if you need to keep something specific private. If you want it just done and working, send Bob a message. Because the previous implementation relies on synchronous sockets, we won’t be able to provide a drop in replacement though.
I’m working through the refactoring myself now. I had already added try/catch around the json parsing even with the old sockets, just because even with valid / expected headers sometimes I got malformed json.
I’m more or less already taking the same approach as you describe above - I was just doing it with the old sockets. Now that I’m forced into using the new sockets instead, I’m finding that they are not at all api compatible, and the differences are just adding that much more friction to the refactoring process.
I’m closing in on having at least this one problematic socket wrangled into submission. For now I’m just ignoring headers and relying on the try/catch around the json parsing (using @Kem Tekinay 's parser for that - it’s MUCH better than the native xojo one).
You should look at the http response codes. If you look at their api docs, theres a list of what the codes mean (I checked this earlier when looking at your case).
I have been, and I do. Unfortunately, this is a pretty young API that is going through some growing pains. It has been my experience that the response codes don’t always match up with what you would expect from them, thus the need to inspect the headers.
For instance, when their api barfs processing one of my requests, nginx has been known to return a nicely formatted HTML error message from their api processor with a response code of 200. Unfortunately, that would cause my json parser to throw an exception.
I’ve wrapped the json parsing in a try/catch, and I’m checking the response codes. In porting my old socket code to the new sockets, it seems like an oversight that I cannot get access to a list of all headers for the response. I guess I’ll file a feature request.
Ugh, I hate that. Ive worked with large and small, young and old APIs and when this type of thing happens I usually file a stern bug report (especially if its somet that you pay for access to) and get them to fix it right away.
For now, if you know what types of requests cause problems, use an app like Paw to duplicate the request and see what the headers would be. Unfortunately if theyre as young as you say, Id expect those headers to change over time as well.
Oh, trust me. I annoy the folks who manage that API more than I pester you.
The API in question was opened earlier this year. We were only their second customer. Yeah, they are pretty new. Headers have changed, and will continue to do so. That’s the whole point of why I need to inspect them at runtime, rather than just periodically using Paw or Postman.
