I have some users with Internet connections that are fast enough on download but much, much slower on upload. For example, satellite cable typically has around 15 Mbps download but only 3 Mbps, or slower, upload and really long latencies of about 500ms or longer. One ISP (AT&T U-verse) also seems to have an extreme amount of speed/latency variability which is not something that is usually quantified in connection speed checkers but seems to be an issue.
Is there anything I can do to support problem connections like this? I know I can preload images but this would only help users with slow download speeds which isn’t the problem. Here are some ideas but I am hoping there are other, better ideas that people would be willing to share:
Provide a connection speed tester on the first page. How hard would it be to implement a speed checker in Xojo (something like what speedtest.net does) on the first page and provide explicit feedback to the user about whether or not their connection will work with this app? I don’t know exactly what’s required by Xojo Web Apps (and I am sure it varies for each app) but I am guessing in the neighborhood of >5 Mbps down/up and 20ms latency would be fine. Are there any bandwidth simulators I can use to verify required bandwidth and latency? [Note: I have tried Fiddler and created custom filters but I can’t get the filters to work (speedtest.net always shows normal speed, not the reduced speed set in Fiddler). The throttling build into Chrome doesn’t allow custom speeds and it doesn’t throttle uploads.]
Minimize the need for fast upload speeds and short latencies. I have no idea what components,events, and actions require fast upload speed and short latency. Are there some components I could eliminate or modify to reduce the need for fast upload speeds and short latencies?
Eliminate the need for good connection by providing the app as a download-only package. I would like to provide this option for people who fail the speed check. I use a lot of youtube video but I could make the videos part of the package. Is this option feasible? I don’t believe it would be feasible for me to create a Desktop version of the app. Instead, what I have in mind is essentially a web app that runs mostly on the local machine. Possible?
Let me know if this isn’t clear. The goal is to support all users, not just the majority of users with fast connections.
A problem I potentially see with this, is to reliably test a user’s speed… you’re going to need a big enough file to substantiate an average speed over a period of at least 10+ seconds. This causes a wait time for the user, and it will require a large chunk of bandwidth on visits. You could limit it, of course, to only ‘test’ on an initial connection (once a cookie is established, bypass the test and store the users speed); however, even then… if you use a 10MB file for testing, and get 100 unique visitors a day… that’s 1GB of data JUST for the test. Over the course of a month, you’ll easily consume 30GB+… (of course, mileage will vary depending on circumstances).
Good point. I suppose I could just provide a link to speedtest.net. I could also make the checker optional. Or, since the app requires downloads anyway, measure the speed while it is doing that. Don’t know how to test upload though.
One problem that’s difficult – if not impossible – to overcome is the latency of the Internet itself, which can vary wildly. I have a 300 Mbps optical fibre connection (fibre all the way to the router on my desk). Many Web sites load almost instantly while others can take a while. Some sites are as slow as they were when I had a 10 Mbps ASDL connection. It all depends on the connections between the remote server and the user.
What you’re running into is the fact that just about everything requires a round trip back to the server. What you need is to push more functionality to the browser and minimize communication to your Xojo code running on the server. Time to start digging into the Web SDK and see if you can create components to use jquery or similar instead of the Xojo event mechanism (which makes heavy use of ajax).
One popular method to reducing latency delays is to reduce the number of transactions required to load your pages, etc.
While there’s likely not much you can do with Xojo-specific server communications, you can reduce the number of images you have.
Google “sprite images and web design” to find out how to take all the little versions of all the little images on your site and combine them into one (or fewer) single graphic image. Then the positioning of that image within your buttons, etc is how you control what is displayed.
Note that this means you’ll need to style your images & icons outside of Xojo’s normal styles and images. You’ll need to use a separate style sheet to control this.
It’s work, but it could be worth it if latency is an issue and you have a lot of little images. If you consider all the separate times the browser has to ask for items, coalescing many images into one could cut that number down by 90% or more in some designs.
Very cool idea! I will give that some thought. Kind of like a gif. I have lots of images so that’s probably a good place for me to focus. Based on these comments, it sounds like image download isn’t as simple as I thought it was; for example, if each download has to communicate back to the server then this might be where some users are encountering bottlenecks. Not sure.
By the way, Fiddler has an add-on that checks images for bloat. I haven’t tried it yet but you can read about Show Image Bloat add-on here.
I have been avoiding looking at the Web SDK. Damn. Guess I better get into it.
I think Fiddler is actually working for my app, it just doesn’t work with speedtest.net so I can’t verify the speeds Fiddler is giving me. I also got Charles to work for my app (but not for speedtest.net; must be something weird about speedtest.net). Charles is more intuitive and you don’t have to mess with custom filters. Both use proxy which I am reading isn’t the best way to do bandwidth limiting. Is there anything better for limiting bandwidth on Windows?
I know it is hard to do, but are there any benchmarks for what the minimum network requirements are for “typical” Xojo Web Apps? I might take the lazy way out and just specify this minimum on the first page and give people a button to go check it if they don’t know or aren’t sure what it is. Low tech but might be helpful for some people.
I don’t know your application but unless you’re exchanging serious amounts of data, I doubt Speed (as in Mbps) is the problem. I don’t think 15Mpbs/3Mbps is bad/slow for a user connection.
I think latency is more of an issue. And for that, why not ping the delay?
As for monitoring http (not https), see httpscoop. Way easier and no need for proxies. (That’s OS X though)
And lastly, I don’t do Xojo Web but can’t you use it with a (free) CloudfLare account to host all your static content like js, images etc.?
[quote=215634:@Marco Hof]I don’t know your application but unless you’re exchanging serious amounts of data, I doubt Speed (as in Mbps) is the problem. I don’t think 15Mpbs/3Mbps is bad/slow for a user connection.
I think latency is more of an issue. And for that, why not ping the delay?
As for monitoring http (not https), see httpscoop. Way easier and no need for proxies. (That’s OS X though)
And lastly, I don’t do Xojo Web but can’t you use it with a (free) CloudfLare account to host all your static content like js, images etc.?[/quote]
If you ping for latency make sure that you increase the frame/packet size from the OS default (OS X 56bytes, Win I believe is 64bytes) Otherwise you will not have the most accurate latency RTT.
For example on OS X to ping a 950 byte packet which is under the MTU of 1500, but large enough to be a real life data packet size - ping -s 950 127.0.0.1
Where exactly is the slowdown? I got the impression from your initial post that your app bogged down while the user was interacting with it. Is that the case, or is it page load times that is the issue?
Another thing I should mention is a tool that comes with Xcode on the Mac
It’s called “Network Link Conditioner” and is installed as a preference pane. It has a number of “preset profiles” that allow the simulation of different network misbehaviors and you can also dial up the latency to see how it affects things (independently of bandwidth)
Latency can be a killer and is tough to detect, simulate and compensate for. It doesn’t matter how fast your bandwidth is if latency is high. All the little connection steps for setting up and tearing down requests for many tiny pieces can really slow things down.
I see that I need to clarify what we are seeing. First, I want to be clear that the only users with problems are those with poor quality connections. One user did a speed test for us using speedtest.net and the latency was 624ms, 21 Mbps download and 1 Mbps upload. I just don’t know if there is anything I can do to get it working correctly for this person but that’s why I am asking the experts. I believe with this bad connection a simple web app wouldn’t even work. Maybe that’s the next test: do a simple app and see what happens.
I can’t replicate the problems that users with poor quality connections are encountering. My connection is 25 Mbps up/down and 20 ms latency so I have to use Fiddler or Charles to dial it down; maybe then I can start to replicate the errors. The app is freezing on different pages for different people. There doesn’t seem to be any specific page that the app is freezing. Sometimes an error pops up. One user got a gradual slowdown then an error that said something like this:
Could not execute returned javascript: ‘WebPage3’ is undefined
Regarding the Network Link Conditioner, I read Keeney’s report on that but it doesn’t help me. I don’t have access to a Mac. However, I do think I can get Fiddler and/or Charles to work. I haven’t gotten around to creating the correct custom settings.
[quote=215701:@Ken Gish]I see that I need to clarify what we are seeing. First, I want to be clear that the only users with problems are those with poor quality connections. One user did a speed test for us using speedtest.net and the latency was 624ms, 21 Mbps download and 1 Mbps upload. I just don’t know if there is anything I can do to get it working correctly for this person but that’s why I am asking the experts. I believe with this bad connection a simple web app wouldn’t even work. Maybe that’s the next test: do a simple app and see what happens.
I can’t replicate the problems that users with poor quality connections are encountering. My connection is 25 Mbps up/down and 20 ms latency so I have to use Fiddler or Charles to dial it down; maybe then I can start to replicate the errors. The app is freezing on different pages for different people. There doesn’t seem to be any specific page that the app is freezing. Sometimes an error pops up. One user got a gradual slowdown then an error that said something like this:
Could not execute returned javascript: ‘WebPage3’ is undefined
Regarding the Network Link Conditioner, I read Keeney’s report on that but it doesn’t help me. I don’t have access to a Mac. However, I do think I can get Fiddler and/or Charles to work. I haven’t gotten around to creating the correct custom settings.[/quote]
Ken this user only has 1mb Upstream and who knows what they were doing to achieve a stellar 624ms RTT That is almost a full second of delay which is extremely bad. I am sure TCP is retransmitting like crazy.
I would start by asking them if they are watching live streaming videos or something that is synchronous in bitrate which may put a dent in that 1mbps upstream.
Ken, take the time to check out Apple’s Network Link Conditioner as Steve mentioned. You can actually simulate bad connections with it and it’s an excellent testing tool as well.
Parallels or VMWare added this capability this year too, I just can’t remember which one.
NetBalancer is the best thing I have tried on Windows. There are many negative reviews for NetBalancer (BSOD, broken Internet connection, requires reinstallation after Windows Updates, high CPU use) but I haven’t experienced any of these problems. It uses something called traffic shaping. In addition to allowing separate custom upload and download speeds, NetBalancer has separate latency settings for upload and download. It also has settings for latency jitter and packet drop rate.
That is almost a full second of delay which is extremely bad. I am sure TCP is retransmitting like crazy.