I know this is a loaded question, but I’ll try to constrain my question… I have a client who needs a Xojo Web app who is concerned about how many connections an app on Xojo Cloud can handle at once. I know the answer will be different for every app…
So, let say the app is the EddiesElectronics demo. How many users could be connected and using the app at once on the larger server with good response times?
I’d like to know the answer to this question as well. I know it is hard to answer but I just need to ballpark it. Assume there is really only one app deployed (and a separate “app” that is just for content like jpg, mp3, png that the main app calls). If you have 256 MB of RAM free (512 MB total available), and it takes 2 MB per connection, does that mean the server can handle about 128 connections? Seems too high. I am guessing Xojo Cloud (Small) can only handle about 50 concurrent users. Is the bottleneck the CPU?
Can/should this be estimated using something like loadimpact.com? Is it safe to use loadimpact.com (for an application that isn’t yet announced; still testing it)?
I have a lot to learn about optimizing but it would be nice to know about resources people use for this purpose.
In our experience you tend to hit the CPU threshold before you hit the RAM ones. RAM is just storing the session data which is not that significant. That is of course if you are not holding 100 images in each session object.
That’s what I was afraid of . It sounds like this might mean it would be hard to ballpark it. The app uses a lot of png, jpg, mp3 plus youtube videos.
I use (loadUI) http://www.loadui.org/… which should be able to handle requests to the Xojo Cloud. Although I can’t speak for the Cloud, I once ran a standalone which was hosted on a ESXI Virtual Machine using only 2GB of RAM on a dual XEON 2GHz processor. It was a simple mortgage rate calculator which handled individual sessions properly and it was able to handle about 500+ connections at once before server responses started getting too extreme to be practical. This was on a 10MB pipe… so I’m confident that was the bottleneck (leaving less that 20Kbps for each connection). In essence, given enough bandwidth and just moderate resources… with proper coding techniques (to handle memory & CPU consumption on calls)… I think Xojo WE can hold up surprisingly well.
Keep in mind that any of these load testing apps may be considered aggressive traffic by the hosting company. If you will be doing load testing, you may want to contact your provider first so you don’t get blocked.
Ok. I probably won’t do it because it doesn’t seem like it would work too well (namely, I can’t see how it would simulate an actual user).
This is a very impressive number. Most sites seldom require that many simultaneous connections.
You could host all of that elsewhere so it won’t be a hit against your cloud server. Youtube already does that for you, so they already don’t count.
For simply hosting of data files, a normal apache server is more efficient.
But consider this: If your code uses in average 20ms to handle an event, your app is limited to 50 connections with one event per second.
And if one user can for example click a button to create a download file, e.g. a PDF and creating the PDF takes 2 seconds, than all other people may get blocked in that time.
So for one client we used here 4 instances of the web app on one server with a script to balance them a bit. This way on a normal day everyone has his own instance and only if more than 4 people come at the same time, they share an instance with others.
That’s a good point. If you used a service like Rackspace’s CloudFiles, the costs are minimal, and the files end up being globally distributed.
I tried loadimpact.com and the output doesn’t make sense. It looks like “User load time (Aggregated (World))” is a roughly constant 300 ms as the number of virtual users is scaled up from 6 to 50. I don’t think this is real or maybe I am misinterpreting the output. Also, I think because the app is cgi, the piecharts showing content type and content type load time distributions are solid. Any way you look at it, loadimpact.com does not seem useful.
What are some good rules-of-thumb that people can use to estimate this stuff? For example, Christian’s idea to use time to handle an event seems like it is on the right track. If you have 50 users and the time to handle an event is 20 ms, you can handle 50 different users’ events in 1 second. How can time per event be estimated? Would the estimate be based on CPU speed?
What criteria are most important in determining when you need to consider doing some sort of optimization? For example, using multiple ports seems to be discussed a lot but I don’t know why it helps; it just seems like another way to chew up finite CPU resources.
Even if you ignore human factors like tolerable lag for the average user, it seems really difficult to estimate server performance. Clearly, it has to be measured because each app is different but when I see measurements like loadimpact.com I don’t have much faith in that approach.
Any help? All I am looking for are ballpark estimates.
Not to confuse the issue even more, but the amount of time spent in an event is not going to be consistent from control to control. It will be highly dependent on what parts of the framework you call in response to user events, and estimating this will be next to impossible.
It’s certainly possible to have a consistent 300ms startup time though. When a session first starts up, we deliver to main parts of the framework to the browser so loading things later will be snappy.
As far as figuring out when it’s time to start scaling your app, the best approach is to think about this early in your design even if you think you’ll have more than 50-100 concurrent users. You can get a huge processing boost by moving processor intensive code into a console app and launch it when necessary (things like sending emails where the process speed is dependent on the speed of the Internet at that time). Beyond that, Load Balancing with multiple instances of your app responding to requests is the next best thing. It does however, get much more complicated if you need the sessions to be able to communicate with one another because you need to set up a mechanism to do that, whether it be through a database or a socket of some kind.
But think about that question very carefully. 50-100 concurrent users. Most people I’ve talked to say “I’m planning to have thousands of users.” But they’re not thinking about them all using the app at exactly the same time.
For my app, it seems unlikely/rare that I would have 50 concurrent users, but it would be nice to get a better idea what my server account can handle (Xojo Cloud Small) for my app. Google Analytics has a Real-Time view which tells me number of active users but I haven’t figured out what, if anything, it is telling me about server performance.
It would be nice if the Xojo Cloud dashboard had statistics like the following that could be plotted for user-selectable time periods (e.g., week/month/year) with min/max/avg options: number of concurrent users, percent CPU usage, percent RAM usage, throughput used, outgoing bandwidth used. These statistics would be very useful and would help people figure out if/what they need to upgrade.
I did a test where I had 10 separate browser windows open (not tabbed as per a previous post I read), distributed across three different browsers (Chrome, Firefox and IE11). and it didn’t crash but the response times were often intolerably slow (30 seconds or more). I think this is primarily testing the limitations of my PC (e.g, it was using most of the 4 GB RAM). My next test will be to have 10 people on different networks get on it at the same time.
There’s got to be an easier way…
You’re right. 30 seconds is crazy, but the only thing I can think of that would do this is if the sessions were fighting for resources. For instance, if it’s a database app, you should have a separate database connection for each session. Also, remember that tight loops lock up all of the sessions. You’ll be better off moving long-running operations to a thread.
Without seeing your app, I can’t really help though.
As far as the Xojo Cloud dashboard goes, those types of statistics are certainly on our to-do list.
I have been told my Xojo Cloud (small) server should be able to handle anywhere between 10 and 50 concurrent users. Not measured, just an educated guess. I have also been told that some day we will have statistics in the dashboard.
A couple things that might help. Google Analytics has Site Speed (in Behavior section). Also, this might help your analysis: http://www.webpagetest.org/
What I told you is that I think your APP should be able to handle 10+ Users. I was not considering your server size at all. Since I do not have access to the source code, the evaluation was done simply by looking at how a browser accesses your app. That being said, the fact that you are not trying to serve those movies, pictures and sounds directly through the app is a good thing. The more you move asset delivery off to the web server, the better, but there is no way tell if your app is doing anything bad internally that might limit concurrent users without actually seeing your code.
Hi all, this is an interesting thread indeed.
I would like to have multiple instances of an app run on a 16 processor virtual machine. If they must be set to different ports do I use a load balancer with port forwarding or have i missed a step?
Also, if I offload media to another server, won’t browsers object to cross site scripting or something like that?
Thanks for your help,
Eric
having images and movies on a normal webserver (e.g. apache) reduces the load to the web app a lot.
You can use a load balancer with port forwarding.
It could be as easy as port = 8000 + (IP address mod 16).