We have a complex web app that runs “OK” in AWS. Not blazingly fast, but not too shabby. But on the client’s internal network, it is slowing to a crawl and we are getting timeout errors. Their network is very fast (ping time from our web server to the endpoints it’s talking to is 1-2 msec) so I don’t think that’s the problem. Similar ping times from client desktops to the server the web app runs on.
Example: we fill a listbox from a JSON endpoint; payload from the JSON server is about 2-3 KB. Sometimes that listbox (which defaults to only 10 items) doesn’t even fill. We also put an image into an imagewell with a URL. When we change that URL, sometimes the imagewell doesn’t change.
Web App is standalone, port 8080, running on Red Hat Enterprise Linux. Currently built with 2017r1, but I’ve also tried 2017r2.
Any suggestions would be appreciated, as the client is getting angry.
@: thanks for the Chrome tip. Doubt that there’s a name issue because they’re using IP addresses. But I will try and decipher the Chrome network dev panel.
@Bob Keeney: I’ve never been able to run a web app in Linux on 32 bit; could never get the missing libraries resolved. So both the AWS and customer servers are 2 core, 64 bit 8 GB RAM. I don’t have any visibility into the servers, but their IT staff says that neither processors nor memory are stressed.
So I’m at a loss here.
I do know that this project takes almost a minute to load on my 2.3Ghz i7 MacBook, and crashes Xojo about twice a week.
Sounds like it could be an unexpected routing or DNS problem.
I know that if we access our local server from an external IP address, forcing the communications to go out our router and back in, it gets very slow sometimes. Accessing the server by its internal IP address is much faster…
@Steve Upton: thanks, but everything is internal to the client. They’re on the same internal network. Ping time from client desktop to server is 1 msec.
@derk: Red Hat Enterprise Linux 7.x, this is the only non-system process on the machine. It’s in a VM, but I believe the disk is 60GB SSD and the RAM is 8 GB.
Perhaps it could be something else. Not sure though as “local” seems to be slower then “aws” it sounds like the machine setup is other (or perhaps network as suggested by @Steve Upton )
@Steve Upton: can access both by IP and hostname, no apparent difference.
This is in a huge datacenter; there are no “normal” machines.
And the host machine is a virtualized Linux server, so no browser is available.
Something that hasn’t been mentioned here is that you may have a “noisy neighbor”. That is virtualized hardware like this is often installed on physical hardware which is configured with 32, 64 or 96GB of RAM and 8 or 16 processor cores and then they cram as many VMs onto the hardware as they can. Depending on what you’re paying, you may only get getting a slice of a virtualized core. If one of the VMs is using a lot of CPU or I/O it can detrimentally affect the other VMs on that machine, effectively making them run really slow… and yes, looking just at your VM, everything looks fine, because yours isn’t doing anything. I’d have the service provider make sure that this isn’t happening.
Another thing you could do is try running “top” using a Shell object within your web app. it’d be essentially the same as doing it from the terminal except that it’s a snapshot of a specific time.
top -n 1 -b
That’s one sample, in batch mode (so you get to see everything, not just one screen’s worth).
@Greg O: interesting. I would hope that their IT people (this is a BIG company, so there are lots of them) would have tools to monitor this kind of “noisy neighbor” problem, but maybe they aren’t looking. I’ll ask.
Running top would only let me see what’s going on in my own VM, though, so that wouldn’t help me detect noisy neighbors. And I’m fairly certain we’re the only process on this VM. It doesn’t even have Apache or MySQL. Just the Xojo standalone web app.
It seems like the problem is happening on container controls that are parsing 4-5 KB of JSON to fill up list boxes (via .addrow) and various other controls. Is there maybe something I should be doing to prevent over-the-wire refreshes on a listbox while it’s being filled?
Yes, but it will let you confirm that the VM isnt overloaded in RAM or CPU. [quote=348376:@Charles Weger]It seems like the problem is happening on container controls that are parsing 4-5 KB of JSON to fill up list boxes (via .addrow) and various other controls. Is there maybe something I should be doing to prevent over-the-wire refreshes on a listbox while it’s being filled?[/quote]
That could be, but then the issue is probably one of pure bandwidth. How many rows/columns of data are we talking about here?
How is the machine virtualized? (Full? (KVM, VMware, VirtualBOX) or Container). How many cores assigned (check /proc/cpuinfo and /proc/meminfo)
Test the network speed with something like iperf (from workstations to server and viceversa).
Also try traceroute in both ways to detect any routing problem as Steve said.
I have some Web apps under CentOS 7, virtualized (KVM, 8Gb, 4 core) and works fine even over Internet, they are CGI’s, and their load on the server is not very high. The physical server is not over loaded.