Some URLConnection requests being throttled

Targets macOS
1

I’m stumped and I need ideas. My URLConnection requests to https://api.nitrado.net are being throttled by the system. I know, it sounds crazy, but hear me out. I have a sample project here that I have tried on an Intel Mac running Ventura, my daily ARM Mac running Sonoma, and a clean VM running Sonoma. They all exhibit this behavior. The first few requests will succeed, then they get stalled by 67 seconds. Yet when I try this on Windows, all 20 requests complete as normal. I have tried both Xojo 2023r2 and 2023r3.

But the plot gets crazier! Looking at console messages, I see some hints that maybe App Nap is to blame. So I use NSProcessInfoMBS to tell the system I’m doing work, and guess what? More requests are allowed before getting throttled! But it still happens. So this has to be macOS doing something.

Here’s the monkeywrench. I’m using a custom URLConnection subclass so that I can have synchronous http requests while still being able to get progress events. This also allows Windows to actually yield correctly. So this means the actual request work is happening on the main thread. The Send method is overridden to pause the calling thread, and ContentReceived will resume the thread. It’s very useful. If I use SendSync on a vanilla URLConnection, there’s no problem. I’d rather not do that because the progress information is very useful. In the sample project, I accidentally left one of my experiments in place, where I sleep the thread in loop instead of pausing it. It made no difference, so I didn’t bother to reupload.

When I try other domains, such as my own or Google, all 20 requests complete fine. So there is something about Nitrado’s API domain - maybe the SSL certificate - that macOS doesn’t like when executing on the main thread.

This situation feels bonkers to me. I’ve been pulling my hair out all day. Does anybody have workaround ideas that don’t involve me switching to SendSync? I’ve tried things like Connection: Close and manually disconnecting the socket when finished, but they made no difference. The demo project is https://stuff.thezaz.com/SynchronousHTTP.zip

2

Correction, SendSync exhibits the same issue. I forgot to include the cache-control: no-cache and connection: close headers in that test.

But that led me to the answer. It’s the connection: close request header. Removing that and letting the server decide, even if it decides the same thing, seems to solve it.

1 Like
3

Have you considered using something else like our CURL classes?

4

I wish I could remember specifics, but I had to switch away from them years ago, because I essentially traded the URLConnection’s problems for CURLSMBS’s problems. Again, I know you like to fix this stuff, but I just can’t remember what it was.