.htaccess and cache-control

Would there be a negative consequence to adding a cache-control line to .htaccess so that users don’t use their stored cache when hitting a Xojo web app?

Here’s the .htaccess file Xojo creates:

Options +ExecCGI order allow,deny deny from all <Files *.cgi> allow from all </Files>

What if I changed it like this:

Options +ExecCGI order allow,deny deny from all <Files *.cgi> allow from all </Files> Header set Cache-Control "no-store"

Besides making the user experience slower and making the app itself more cpu intensive?

Well, what if I use

Header set Cache-Control "no-cache"

Same user experience decrease and cpu intensity increase? Though I wonder if that won’t do anything because I don’t set ETag, and I think that’s what it looks at. No?

The reason I ask this stuff is that I notice that the few users who get an Internal Server Error 500 get it every time they hit the Web app when it’s not running (causing a Line 118 error in the CGI file). I’m assuming they have a browser cache that’s causing this. Always the same users. The immediate subsequent time they hit the app, it’s fine. Besides asking them to clear their browser cache, I thought the above cache-control line might stop that behavior.

The web framework does though.

I wouldn’t penalize all of your users because of a few. Perhaps instead, set app.autoquit to false and then do your own logic during the least busy time of the day and quit when there are no active sessions. Then the only time this should happen is if one of the few users is the first person of the day to hit your app.

Thanks, Greg.

What does the framework do with the ETag? I’m worried that the current ETag would match the ETag the user has in their browser cache from when they first got the Error 500, so the cache won’t be refreshed. Unless you’re saying the framework updates the ETag frequently, not just when the Web app is built and deployed.

Well, quite a while ago I was getting calls that users were getting persistent, albeit infrequent, Error 500s. When that would happen, I’d have to go online, manually rename the CGI file, wait a few seconds, and then rename it back. This would cause the app to quit, using Phillip Zedalis’ popular App Timer technique that kills the app when the CGI file can’t be found.

I ended all this by changing that App Timer code to also kill the app when it sees no Sessions (I found Auto.Quit didn’t reliably kill the app when Error 500 was present). Unfortunately, I think this results in more frequent Error 500s because it results in the app frequently not running, allowing users with a bad cache to trigger it more often. But this is outweighed by the fact that I have never had to intervene on a persistent Error 500 since doing this. Now when a user gets an Error 500, they just reload and it’s gone.

So the upshot is that I don’t want to go back to persistent, albeit infrequent, Error 500s, which I think will happen if the app stays awake when no Sessions are present after an Error 500 is triggered.

Etags are hashes of the items they are delivering. If they change, so does the hash and the browser gets the new one. If not, the browser keeps what it’s got.

As for the autoquit thing, if your app doesn’t leak sessions and doesn’t get itself into any tight loops that it can’t escape from and doesn’t run out of memory on the server, turning off autoquit shouldn’t hurt you. On the other hand, any one of these combined with autoquit could cause the app to get into a state where it can’t respond.

The only other thing I can think of is that these users have some sort of caching proxy on their network or computer which is trying to be smart by ignoring the caching rules from the server.

So, when these users have an Internal Server Error 500 in their browser cache, I assume the Etags they have in their browser cache could match the Etags on the server because those are the Etags they got when they first got that error. So forcing a a re-validation would not help (i.e., doing a no-cache). Unless I really don’t understand how Etags work.

Aside from turning off AutoQuit, would a good compromise be limiting the max-age of the cache appropriately?

Actually, now that I read more on max-age, I doubt that would help if the Etags in the user’s Error 500 cache match those on the server. If they match, the bad cache will be used and the max-age period will just be renewed

The error 500 is from the server itself. It’s an indication that something unexpected went wrong.

The etags are used to let the browser not download a resource that it already has. Basically, the browser says to the server “I need this file, the one I have has this signature”. If the signature matches the one the server has, it responds by saying Not Modified telling the browser to use the one it has. If not, the server sends the new one.

I seriously doubt that the connection issues are due to caching… at least in this sense.

Well, the user that’s getting these periodic Error 500s just told me she indeed cleared her iPhone Safari cache when I asked to think about doing that a week or so ago, and the errors continued. So I guess you’re right and this isn’t a cache problem. I just couldn’t figure out what else it could be since it seems isolated to one or two users, and very consistent with them. In any event, they can always get around the error by immediately reloading, so it’s more of a nuisance than anything else. Good luck with your XDC talks.

Ah! You hadn’t mentioned that one was on an iPhone… are both of them?

JavaScript on mobile Safari is a very special thing that we have been battling with for years. The general gist of the problem is that when a user dismisses the Safari app, they aren’t actually closing Safari. It continues to run for a short time, then shifts into something similar to macOS sleep mode for a bit (which recovers just fine) and then goes to what I refer to as “deep sleep” in which all execution stops and is not restarted when it wakes. When the user comes back, the app appears to still be running because the last screen is still showing, but there’s no longer a network connection to the back end and when the user does the first thing that sends a message to the server, BOOM. It’s not so much that the server app isn’t running, but that what was sent is completely out of date and the server doesn’t know what to do with it.

Yes. These users are iPhone users, as are almost all the users of this app.

So this can result in an Internal Server Error 500?

I think (but am not sure yet) that these problems arise when the mobile user uses a Home Screen Shortcut to load the URL, and not when they use a Browser Bookmark.

On my Android I found the above to be true, using Chrome.

Does iPhone Chrome share the same Javascript problem?

[quote=434359:@Greg O’Lone]Ah! You hadn’t mentioned that one was on an iPhone… are both of them?

JavaScript on mobile Safari is a very special thing that we have been battling with for years. The general gist of the problem is that when a user dismisses the Safari app, they aren’t actually closing Safari. It continues to run for a short time, then shifts into something similar to macOS sleep mode for a bit (which recovers just fine) and then goes to what I refer to as “deep sleep” in which all execution stops and is not restarted when it wakes. When the user comes back, the app appears to still be running because the last screen is still showing, but there’s no longer a network connection to the back end and when the user does the first thing that sends a message to the server, BOOM. It’s not so much that the server app isn’t running, but that what was sent is completely out of date and the server doesn’t know what to do with it.[/quote]

I only rarely see an Internal Server Error 500 with a desktop user. I don’t remember the last time I saw that. When it happens, it seems confined to mobile users, and predominately iPhone users (occasionally AndroidPhone users). Right now, it’s confined to the same iPhone user consistently. Their most recent server 500 log entry shows this:

After that, they get right in on a reload.

Do these problems seem absent with Standalone Web apps?

Does Web API 2.0 deal with this known problem with JavaScript on mobile Safari?

Let’s tackle these one at a time.

Event.TimedOut means that you have set Session.Timeout to some number other than zero. It would be helpful to know what that is.

[quote=436433:@Greg O’Lone]Let’s tackle these one at a time.

Event.TimedOut means that you have set Session.Timeout to some number other than zero. It would be helpful to know what that is.[/quote]
Ok. Well, Session.Timeout will be either 30 or 900, depending on whether their Session was at the LoginPage and they never bothered to log in (Timeout is 30 seconds), or they logged in and are in what we can call the MainPage, where they can now enter some data if they want (Timeout is 15 minutes).

The Session.TimedOut code is simple:

Self.Quit

Looking at the access.log, the last event from that IP Address was about 36 hours prior. Here’s the prior non-500 event, followed by the 500 event I posted above, with all intervening events from other IP Addresses removed (GMT timezone) and irrelevant data removed:

[quote][13/May/2019:03:50:18 +0000] . . . GET . . . /comm/serverevent HTTP/1.1" 200 10
[14/May/2019:16:02:40 +0000] . . . POST . . . /comm/event/Event.TimedOut HTTP/1.1" 500 649[/quote]

I don’t usually see that TimedOut event in a 500 log entry. That just happened to be the last one, and it was all by itself, without any 500 events preceding or following it. When I see a 500 event, I usually see something like

But it can be just about anything.

And these 500 events are not immediately preceded by other events by the same IP Address, unless those immediately preceding events are also 500 events from the same IP Address. So such an event (or series of a few such events from the same IP Address) seems to always arise when the user simply hits the app. And they are always associated with those Line 118 errors in the CGI file that many have reported, so I assume it’s always when the user hits the app when it’s not running.

I’m assuming this all has to do with the mobile Javascript problem you mentioned before.

Edit:

And, BTW, dropping the LoginPage timeout to 30 seconds is fairly new. Users were previously always timed out at 15 minutes before, and I was getting the same behavior then.

And should it be relevant, I just noticed an Error 500 for a desktop user (Macintosh Chrome). These are rare. The following are access.log entries for that. The first entry doesn’t have an error 500, and the next three do:

[quote]. . . [15/May/2019:05:16:01 +0000] “GET . . . /comm/serverevent HTTP/1.1” 200 10 . . .
. . . [15/May/2019:05:31:28 +0000] “GET . . . /comm/serverevent HTTP/1.1” 500 649 . .
. . . [15/May/2019:05:31:28 +0000] “GET . . . /framework/pagestop@2x.png HTTP/1.1” 500 649 . .
. . . [15/May/2019:05:31:28 +0000] “GET . . . /xojo/ping HTTP/1.1” 500 649 . .[/quote]

15 minutes ticked off between the first event and the second one, and no other user hit the app during that 15-minute period. And, of course, those three error 500 events are associated with line 118 problems in the CGI file. After all that, that IP Address reloaded fine.

Error 500 when associated with the cgi script usually means that the app itself is either not responding fast enough or that it outright crashed.

And you have a 15 minute timeout. That’s probably relevant.

Unfortunately none of the stuff you are describing is out of the ordinary as far as I can tell except that the clients are sometimes getting 500 errors. But as I said, that could just be an exception that’s causing the app to crash.

[quote=436483:@Greg O’Lone]Error 500 when associated with the cgi script usually means that the app itself is either not responding fast enough or that it outright crashed.

And you have a 15 minute timeout. That’s probably relevant.

Unfortunately none of the stuff you are describing is out of the ordinary as far as I can tell except that the clients are sometimes getting 500 errors. But as I said, that could just be an exception that’s causing the app to crash.[/quote]
Given my user logs, what happened after that 15-minute timeout is their browser auto reloaded and then got an error 500. Looking at my access.log this morning, that happened a few more times, and then it stopped happening. The user appears to have not closed that browser window and left their computer, resulting in these browser auto reloads. I submitted a Feedback that was verified a while ago that shows a simple app with no important code will eventually result in CGI Line 118 errors when left to run when the user leaves their browser window running: <https://xojo.com/issue/54665>

Regarding an app exception, I have Unhandled App Exceptions written to a text file, and none appear. I have also tested this to make sure such a text file can be written on such an exception, and it can.

All this said, the user can always hit the app successfully the next time they reload their browser after getting an error 500, so this is more of a minor annoyance to the user than a major problem.