I’m running into an odd issue and I can’t seem to figure it out. I’ve created a REST API with Xojo using the HandleURL. My app is functioning fine until I request large datasets that takes time to organize into JSON. I’ve used POSTMAN, PHP CURL, JQUERY to test my API.
If I run the app in debug, call the GET with POSTMAN it will wait for a second and return with “Error: socket hang up”. Then a second later I can see the API finishing the response. I’m getting simliar errors with JQUERY/CURL (No response). I’ve tried sending a response status of 102/100. I’ve set the timeout for POSTMAN to infinite and 2 minutes.
For context this method/query is taking about 16 seconds to gather the data in Xojo and respond. POSTMAN seems to return with “Error: socket hang up” at about the 9-11 second mark. I checked with the firewall team and it seems to be functioning correctly. I’m currently out of ideas, I’m all ears!
Using Xojo Release 2023r1.1
I’ve also tried it in Release 2022r4.1
Hey Ricardo, I’ll add some info I’ve discovered that may help.
I optimized the view. This seemed to allow more records sometimes.
limited the pull to 500 records. This was to minimize the error happening.
My code originally used a prepared statement to get the parent JSON. I then looped through that JSON to get all child elements. The error always happened inside my loop.
//Code to get JSON
For t As Integer = 0 To JSON.Count - 1 //Here is when I got child records for each parent. //After waiting some time the Socket Hang up error occurred.
Next
Admittedly, the child records could be more than 5 per record. It could potentially add up to thousands.
HandleURL runs on the main thread and blocks everything else, so you should definitely use a thread for iterating the records and creating a response, but the other thing I’m going to suggest is that you make it so the client can make a request and get a token or a temporary url back which tells the client how to check and see if the response is ready, because locking your app up for 16 seconds is going to cause all kinds of problems for you.
Also, make sure you have your threads yield time to everything else so the app will keep running, especially if it’s running as a daemon (like on Xojo cloud) because if the OS doesn’t get periodic pings from the app, it may assume that the app died or hung and just restart it.
Hey Greg, do you have any suggestions for handling the data with a thread? I currently have methods in the HandleURL which I pass values to. These methods return JSON that I write as WebResponse.
I can’t seem to figure out a clean way to get data to the method/thread and return. The only thing I can come up with is using session/app properties? That seems like it would be a bad idea. Also not sure how (or if) that would work?
@Ezekiel_Burke could you please open a new Issue with a sample project? I’ve just fixed a similar issue and I’d like to double check it fixes yours, or if I need to fix something else for next release.
Let me try to elaborate a little on Greg’s design description to see if that helps.
For long running queries and processes, it is standard to instantly return some method of allowing the client to check to see if the process has completed (a tracking ID of some sort). Your server sets off to do-the-thing and the client periodically checks the progress endpoint until that says the result is ready.
When the result is ready, the client fetches the result. This is a multiple step, multiple request process.
You won’t be able to use a session anyway without more work. Requests that come into HandleURL are not connected to a particular session.
When I’m doing APIs in Xojo, I typically create a module with its own HandleURL method with the same signature as the one on App and just shuffle all of the valid api requests over there. Once there you can split out the requests into finer detail methods to keep down the clutter and to pull data, temporarily Store it for a later release request by the client. Since modules can contain classes, just put your thread classes right in there and everything will be easy to find.