handleURL handleSpecialURL problem with Safari

Hi folks. I was having a very annoying problem where Safari would drop the next update or event from my web app if the previous resource was something I served up through the handleURL or handleSpecialURL. The same thing did not appear to be a problem in Chrome or Firefox and I didn’t test with anything else. I am serving up some thumbnail images that change fairly rapidly and their onLoad handler in the javascript tries to call Xojo.triggerServerEvent to send that data back to my app and I would miss a high percentage of those, or other events being sent back to the server. The console in Safari’s web info panel would just scroll errors up as fast as I could load images. I never got those events or the other events that would error out.

It turns out that Xojo’s built in web server does not support Keep-Alive connections. Every connection coming from the server under it’s own control adds the “Connection: Close" header so that the browser will not try to re-use the connection. The default for http 1.1 however is to keep alive. That header is not added automatically by Xojo when you use the handleURL or handleSpecialURL functions. Safari would happily try to re-use that socket only to discover that it was closed and would log a “the network connection was lost” error and then rather than try again with a new socket like Chrome did it just gives up.

The best solution would be for Xojo to update their web server to support persistent connections :wink: That would probably speed up complicated things quite a bit actually, I’ll put in a feature request for that shortly. In the meantime you must add a Connection: Close header to your outgoing request or Safari will choke on the very next item it tries to get or send to the server. I was actually watching the framework get a refused connection for it’s “ping” command and reloading itself repeatedly over and over sometimes it would get stuck in a loop and never actually finish loading the page. Or just randomly reload later when the ping request happened to come right after loading something from the handleURL event!

So do something like:

request.header( “Connection”) = “close”

before you return true from either of those handlers to make sure that Safari works with your web apps!

That’s a major overhaul, and yes, there’s already a feature request for it. The built-in Server is currently only http/1.0 compliant.

Please do put in a bug report though. Connection: Close should definitely be in there.

Yes, the improvement of the web server is long overdue. Compared to today’s technologies, Xojo is very late. It’s depressing.

For the front-end, we can manage. With the webSdk, in particular, we can overcome some of the controls and features missing from the framework.

But for the back-end, we are completely dependent on the built-in web server.

Also, in addition to the “Xojo Web” framework, I think that an “Xojo Web API” version would be interesting for Xojo Inc., even commercially speaking.

This version will only be centered around the handleURL, with optimizations, and without the web framework.

[quote=352519:@olivier v]Also, in addition to the “Xojo Web” framework, I think that an “Xojo Web API” version would be interesting for Xojo Inc., even commercially speaking.

This version will only be centered around the handleURL, with optimizations, and without the web framework.[/quote]
You can already do this with the existing system. Just make sure you intercept the case of an empty path in App.HandleURL and no sessions will ever be created.

Of course, but what about optimizations?

https://forum.xojo.com/26481-xojo-vs-php-speed/ :

[quote]The bottleneck is not 64 bit, it’s the web app framework. I rebuilt the whole thing using raw TCPSocket and ServerSocket, parsing and forming the HTTP myself. Now Xojo is slightly (10-20 msec per call) faster than PHP. Woohoo!

So the web app framework is about four to six times slower than raw sockets for my endpoints-only use case.

[/quote]

Or use the CGIApplication template

I have built a HTTP 1.1 compliant web server in Xojo. It is very fast. I am not sure I will open source it as it has limited appeal in terms of market.

As for @James Sentman issue with Safari I have never seen that. The “Connection: keep-alive” header is sent by the client and the server responds back that it can handle it. The Xojo HTTP 1.0 complaint web server would not respond back saying it can keep alive connections plus it responds back with a 1.0 header also informing the browser that it cannot handle it.

James are you sure you are not adding additional headers or manipulating the headers to make the server seem more 1.1 compliant? I saw your previous post regarding video streaming so perhaps the browser is assuming it is 1.1 compliant which it is not.

Let me put it another way: if this was truly an issue with Safari and Xojo Web then every web app would be broken with Safari.

[quote=352568:@Phillip Zedalis]James are you sure you are not adding additional headers or manipulating the headers to make the server seem more 1.1 compliant? I saw your previous post regarding video streaming so perhaps the browser is assuming it is 1.1 compliant which it is not.

Let me put it another way: if this was truly an issue with Safari and Xojo Web then every web app would be broken with Safari.[/quote]

Only those that use the HandlURL system and do not specifically add the connection close header. I was initially adding in the connection: Keep-Alive header as I had misread the headers display of what Xojo was sending. Thats how I found the problem in the first place was comparing my headers to those sent by a regular request to the built in server. So I removed the keep-alive but that didn’t solve the problem. I had to actually add connection: close to solve the problem. I have not actually verified that Safari defaults to an http 1.0 with keep alive if you don’t specify connection-close, but that is what it seemed like. In the meantime specifically adding connection close like the xojo server does for every other connection solves the problem. I could be crazy though and not have removed it in every case as I was testing. I’ll go back and double check that later on.

I did have the same problem with the short video snippet streaming system, but I use the handleURL system for much more than that.

I can do some testing but I suspect the header had to be added somewhere. The Xojo web server will respond with HTTP 1.0 in all circumstances. In HTTP 1.0 the keep-alive property can be requested by client but the server has to respond back saying it supports it or it will not assume it can.

All browsers want to default to HTTP 1.1 with keep-alive active but should gracefully downgrade when it detects the server does not support 1.1 features.

Are you on High Sierra? I have not tested Safari there so perhaps it is a regression.

Another thing to keep in mind is if ANY of your requests from HandleURL request keep-alive or 1.1 specific features then it is possible the browser is elevating its internal understanding of the web server as a 1.1 server vs 1.0 causing subsequent requests to fail as it attempts keep-alive. This would correlate with your assessment that some of the requests were working and others were not.

I have verified that by removing the “Connection: close” header that Safari assumes keep-alive and dumps errors rather than closing and recreating a connection. It is definitely assuming that the connection will stay open unless specified to close. If I simply remove the connection: close header addition line I get the exact same flood of errors that I get if I specifically specify connection: keep-alive”

I also tested with removing all my other header additions that I added to try to keep it from caching individual frames and the like, and that didn’t make any difference. Safari definitely assumes a keep alive connection if you don’t specifically specify a connection-close.

I get the same errors and lost “ping” or other events if I fail to specify the connection close event, even if I remove every other header addition. Safari is definitely doing something goofy there with http/1.0 connections that it seems you can only fix by adding the connection close header.

I tested this with Safari on High Sierra and just regular Sierra. I didn’t test further back than that. But the problem has been around for a while then. In my case the ping connections that the web framework makes are often refused causing the entire page to reload over and over again at some random intervals.

So are all of these tests done with your code that sends back “range” responses? I ask because that’s definitely an http/1.1 thing and you may be confusing Safari with those headers.

nope, no range involved in this test at all. I specifically removed that so it wouldn’t affect the test.