.htaccess and cache-control

  1. 3 weeks ago

    Ralph A

    Apr 27 Santa Monica, California

    Would there be a negative consequence to adding a cache-control line to .htaccess so that users don't use their stored cache when hitting a Xojo web app?

    Here's the .htaccess file Xojo creates:

    Options +ExecCGI
    order allow,deny
    deny from all
    <Files *.cgi>
    allow from all
    </Files>

    What if I changed it like this:

    Options +ExecCGI
    order allow,deny
    deny from all
    <Files *.cgi>
    allow from all
    </Files>
    Header set Cache-Control "no-store"
  2. Greg O

    Apr 27 Xojo Inc

    Besides making the user experience slower and making the app itself more cpu intensive?

  3. Ralph A

    Apr 27 Santa Monica, California

    Well, what if I use

    Header set Cache-Control "no-cache"

    Same user experience decrease and cpu intensity increase? Though I wonder if that won't do anything because I don't set ETag, and I think that's what it looks at. No?

    The reason I ask this stuff is that I notice that the few users who get an Internal Server Error 500 get it every time they hit the Web app when it's not running (causing a Line 118 error in the CGI file). I'm assuming they have a browser cache that's causing this. Always the same users. The immediate subsequent time they hit the app, it's fine. Besides asking them to clear their browser cache, I thought the above cache-control line might stop that behavior.

  4. Greg O

    Apr 27 Xojo Inc

    @Ralph A Though I wonder if that won't do anything because I don't set ETag, and I think that's what it looks at. No?

    The web framework does though.

    The reason I ask this stuff is that I notice that the few users who get an Internal Server Error 500 get it every time they hit the Web app when it's not running (causing a Line 118 error in the CGI file). I'm assuming they have a browser cache that's causing this. Always the same users. The immediate subsequent time they hit the app, it's fine. Besides asking them to clear their browser cache, I thought the above cache-control line might stop that behavior.

    I wouldn’t penalize all of your users because of a few. Perhaps instead, set app.autoquit to false and then do your own logic during the least busy time of the day and quit when there are no active sessions. Then the only time this should happen is if one of the few users is the first person of the day to hit your app.

  5. Ralph A

    Apr 27 Santa Monica, California

    Thanks, Greg.

    The web framework does though.

    What does the framework do with the ETag? I'm worried that the current ETag would match the ETag the user has in their browser cache from when they first got the Error 500, so the cache won't be refreshed. Unless you're saying the framework updates the ETag frequently, not just when the Web app is built and deployed.

    Perhaps instead, set app.autoquit to false and then do your own logic during the least busy time of the day and quit when there are no active sessions.

    Well, quite a while ago I was getting calls that users were getting persistent, albeit infrequent, Error 500s. When that would happen, I'd have to go online, manually rename the CGI file, wait a few seconds, and then rename it back. This would cause the app to quit, using Phillip Zedalis' popular App Timer technique that kills the app when the CGI file can't be found.

    I ended all this by changing that App Timer code to also kill the app when it sees no Sessions (I found Auto.Quit didn't reliably kill the app when Error 500 was present). Unfortunately, I think this results in more frequent Error 500s because it results in the app frequently not running, allowing users with a bad cache to trigger it more often. But this is outweighed by the fact that I have never had to intervene on a persistent Error 500 since doing this. Now when a user gets an Error 500, they just reload and it's gone.

    So the upshot is that I don't want to go back to persistent, albeit infrequent, Error 500s, which I think will happen if the app stays awake when no Sessions are present after an Error 500 is triggered.

  6. Greg O

    Apr 28 Xojo Inc

    Etags are hashes of the items they are delivering. If they change, so does the hash and the browser gets the new one. If not, the browser keeps what it’s got.

    As for the autoquit thing, if your app doesn’t leak sessions and doesn’t get itself into any tight loops that it can’t escape from and doesn’t run out of memory on the server, turning off autoquit shouldn’t hurt you. On the other hand, any one of these combined with autoquit could cause the app to get into a state where it can’t respond.

    The only other thing I can think of is that these users have some sort of caching proxy on their network or computer which is trying to be smart by ignoring the caching rules from the server.

  7. Ralph A

    Apr 28 Santa Monica, California

    Etags are hashes of the items they are delivering. If they change, so does the hash and the browser gets the new one. If not, the browser keeps what it’s got.

    So, when these users have an Internal Server Error 500 in their browser cache, I assume the Etags they have in their browser cache could match the Etags on the server because those are the Etags they got when they first got that error. So forcing a a re-validation would not help (i.e., doing a no-cache). Unless I really don't understand how Etags work.

    Aside from turning off AutoQuit, would a good compromise be limiting the max-age of the cache appropriately?

  8. Ralph A

    Apr 28 Santa Monica, California

    Aside from turning off AutoQuit, would a good compromise be limiting the max-age of the cache appropriately?

    Actually, now that I read more on max-age, I doubt that would help if the Etags in the user's Error 500 cache match those on the server. If they match, the bad cache will be used and the max-age period will just be renewed

  9. Greg O

    Apr 28 Xojo Inc

    The error 500 is from the server itself. It’s an indication that something unexpected went wrong.

    The etags are used to let the browser not download a resource that it already has. Basically, the browser says to the server “I need this file, the one I have has this signature”. If the signature matches the one the server has, it responds by saying Not Modified telling the browser to use the one it has. If not, the server sends the new one.

    I seriously doubt that the connection issues are due to caching... at least in this sense.

  10. Ralph A

    Apr 28 Santa Monica, California

    Well, the user that's getting these periodic Error 500s just told me she indeed cleared her iPhone Safari cache when I asked to think about doing that a week or so ago, and the errors continued. So I guess you're right and this isn't a cache problem. I just couldn't figure out what else it could be since it seems isolated to one or two users, and very consistent with them. In any event, they can always get around the error by immediately reloading, so it's more of a nuisance than anything else. Good luck with your XDC talks.

  11. Greg O

    Apr 29 Xojo Inc

    Ah! You hadn’t mentioned that one was on an iPhone... are both of them?

    JavaScript on mobile Safari is a very special thing that we have been battling with for years. The general gist of the problem is that when a user dismisses the Safari app, they aren’t actually closing Safari. It continues to run for a short time, then shifts into something similar to macOS sleep mode for a bit (which recovers just fine) and then goes to what I refer to as “deep sleep” in which all execution stops and is not restarted when it wakes. When the user comes back, the app appears to still be running because the last screen is still showing, but there’s no longer a network connection to the back end and when the user does the first thing that sends a message to the server, BOOM. It’s not so much that the server app isn’t running, but that what was sent is completely out of date and the server doesn’t know what to do with it.

  12. Ralph A

    Apr 29 Santa Monica, California
    Edited 3 weeks ago

    Yes. These users are iPhone users, as are almost all the users of this app.

    So this can result in an Internal Server Error 500?

    I think (but am not sure yet) that these problems arise when the mobile user uses a Home Screen Shortcut to load the URL, and not when they use a Browser Bookmark.

    On my Android I found the above to be true, using Chrome.

  13. Ralph A

    Apr 29 Santa Monica, California

    Does iPhone Chrome share the same Javascript problem?

  14. 5 days ago

    Ralph A

    May 14 Santa Monica, California

    @Greg OLone Ah! You hadn’t mentioned that one was on an iPhone... are both of them?

    JavaScript on mobile Safari is a very special thing that we have been battling with for years. The general gist of the problem is that when a user dismisses the Safari app, they aren’t actually closing Safari. It continues to run for a short time, then shifts into something similar to macOS sleep mode for a bit (which recovers just fine) and then goes to what I refer to as “deep sleep” in which all execution stops and is not restarted when it wakes. When the user comes back, the app appears to still be running because the last screen is still showing, but there’s no longer a network connection to the back end and when the user does the first thing that sends a message to the server, BOOM. It’s not so much that the server app isn’t running, but that what was sent is completely out of date and the server doesn’t know what to do with it.

    I only rarely see an Internal Server Error 500 with a desktop user. I don't remember the last time I saw that. When it happens, it seems confined to mobile users, and predominately iPhone users (occasionally AndroidPhone users). Right now, it's confined to the same iPhone user consistently. Their most recent server 500 log entry shows this:

    . . . POST . . . /comm/event/Event.TimedOut HTTP/1.1" 500 649 . . .

    After that, they get right in on a reload.

    Do these problems seem absent with Standalone Web apps?

    Does Web API 2.0 deal with this known problem with JavaScript on mobile Safari?

  15. Greg O

    May 14 Xojo Inc

    Let’s tackle these one at a time.

    Event.TimedOut means that you have set Session.Timeout to some number other than zero. It would be helpful to know what that is.

  16. Ralph A

    May 14 Santa Monica, California
    Edited 5 days ago

    @Greg OLone Let’s tackle these one at a time.

    Event.TimedOut means that you have set Session.Timeout to some number other than zero. It would be helpful to know what that is.

    Ok. Well, Session.Timeout will be either 30 or 900, depending on whether their Session was at the LoginPage and they never bothered to log in (Timeout is 30 seconds), or they logged in and are in what we can call the MainPage, where they can now enter some data if they want (Timeout is 15 minutes).

    The Session.TimedOut code is simple:

    Self.Quit

    Looking at the access.log, the last event from that IP Address was about 36 hours prior. Here's the prior non-500 event, followed by the 500 event I posted above, with all intervening events from other IP Addresses removed (GMT timezone) and irrelevant data removed:

    [13/May/2019:03:50:18 +0000] . . . GET . . . /comm/serverevent HTTP/1.1" 200 10
    [14/May/2019:16:02:40 +0000] . . . POST . . . /comm/event/Event.TimedOut HTTP/1.1" 500 649

    I don't usually see that TimedOut event in a 500 log entry. That just happened to be the last one, and it was all by itself, without any 500 events preceding or following it. When I see a 500 event, I usually see something like

    . . . framework/appicon128.png HTTP/1.1" 500

    But it can be just about anything.

    And these 500 events are not immediately preceded by other events by the same IP Address, unless those immediately preceding events are also 500 events from the same IP Address. So such an event (or series of a few such events from the same IP Address) seems to always arise when the user simply hits the app. And they are always associated with those Line 118 errors in the CGI file that many have reported, so I assume it's always when the user hits the app when it's not running.

    I'm assuming this all has to do with the mobile Javascript problem you mentioned before.

    Edit:

    And, BTW, dropping the LoginPage timeout to 30 seconds is fairly new. Users were previously always timed out at 15 minutes before, and I was getting the same behavior then.

  17. Ralph A

    May 14 Santa Monica, California

    And should it be relevant, I just noticed an Error 500 for a desktop user (Macintosh Chrome). These are rare. The following are access.log entries for that. The first entry doesn't have an error 500, and the next three do:

    . . . [15/May/2019:05:16:01 +0000] "GET . . . /comm/serverevent HTTP/1.1" 200 10 . . .
    . . . [15/May/2019:05:31:28 +0000] "GET . . . /comm/serverevent HTTP/1.1" 500 649 . .
    . . . [15/May/2019:05:31:28 +0000] "GET . . . /framework/pagestop@2x.png HTTP/1.1" 500 649 . .
    . . . [15/May/2019:05:31:28 +0000] "GET . . . /xojo/ping HTTP/1.1" 500 649 . .

    15 minutes ticked off between the first event and the second one, and no other user hit the app during that 15-minute period. And, of course, those three error 500 events are associated with line 118 problems in the CGI file. After all that, that IP Address reloaded fine.

  18. Greg O

    May 15 Xojo Inc

    Error 500 when associated with the cgi script usually means that the app itself is either not responding fast enough or that it outright crashed.

    15 minutes ticked off between the first event and the second one, and no other user hit the app during that 15-minute period.

    And you have a 15 minute timeout. That’s probably relevant.

    Unfortunately none of the stuff you are describing is out of the ordinary as far as I can tell except that the clients are sometimes getting 500 errors. But as I said, that could just be an exception that’s causing the app to crash.

  19. Ralph A

    May 15 Santa Monica, California

    @Greg OLone Error 500 when associated with the cgi script usually means that the app itself is either not responding fast enough or that it outright crashed.

    And you have a 15 minute timeout. That’s probably relevant.

    Unfortunately none of the stuff you are describing is out of the ordinary as far as I can tell except that the clients are sometimes getting 500 errors. But as I said, that could just be an exception that’s causing the app to crash.

    Given my user logs, what happened after that 15-minute timeout is their browser auto reloaded and then got an error 500. Looking at my access.log this morning, that happened a few more times, and then it stopped happening. The user appears to have not closed that browser window and left their computer, resulting in these browser auto reloads. I submitted a Feedback that was verified a while ago that shows a simple app with no important code will eventually result in CGI Line 118 errors when left to run when the user leaves their browser window running: Feedback Case #54665

    Regarding an app exception, I have Unhandled App Exceptions written to a text file, and none appear. I have also tested this to make sure such a text file can be written on such an exception, and it can.

    All this said, the user can always hit the app successfully the next time they reload their browser after getting an error 500, so this is more of a minor annoyance to the user than a major problem.

or Sign Up to reply!