NGINX - Proxy making me crazy

I have been trying to get NGINX running on Windows Server 2008 r2 to work as a proxy.

I thought … I say thought … I saw it work a few times but I needed a few more changes and now it will not work.

Did I mention this is making me crazy?

Currently I get a badly formatted:

The application has gone off-line. Please try again later. This application has encountered an error and cannot continue.

Maybe due to session persistence or a route problem on different IP addresses on the server (it has 3 IP addresses).

My goal is to use a single domain name on port 443 (HTTPS) to launch several backend stand alone apps, adding a dummy virtual path at the end of the URL. I cannot use port numbers directly because I am behind a network firewall and I don’t have any control of the settings.

I have positively confirmed that is is coming from the specific stand alone app by shutting it down and getting no response. When the app is running the response is almost instant.

Oddly I have, on a totally different Windows Server 2008 r2 box, gotten this to work using Sub-Domains to route to the proper app all on the same public IP. I have just about duplicated the config from this server and I just get the badly formatted “gone off-line” message.

My current testing has been inside the network firewall and not on standard ports because I cannot take down port 80 until I have a solution to move all of the CGI Web apps to standalone. For now I have abandoned trying to configure multiple different Web apps for testing just to keep the test case very simple. Right now I just want to hit port 8083 and proxy to the running stand alone app on 8443. Windows firewall is open on all of the relevant ports.

Just for fun I pointed the proxied server to a public Website and it returns a mis-formatted page but the correct domain.

At this point I don’t know what I do know or don’t know. Proxy, reverse proxy …

Did I mention this is making me crazy?

I’d consider hiring a Nginx pro or purchasing their commercial product with support. Reverse proxies mostly work the same way and it has little to do with Xojo. You can technically do it with IIS as well.

@Mark Strickland: I’ve been running NGINX for a few weeks now, configured as a reverse proxy, to serve up multiple Xojo standalone Web apps on an Amazon Lightsail server.

The config file has been stripped down to the absolute minimum that is needed to make it work. I’d be happy to share it with you.

If you’re interested, let me know.

It shouldn’t be that hard. I say take Tim up on his generous offer and then if you’re still having problems post and update with details and we’ll take another run at it.

Tim,

Yes I would love to have a copy of your configuration file.

I have stripped mine down to the minimum just for testing but I must be missing some setting.

I will send you a private message with my email address.

Thanks.

we had better luck with haproxy. if i remember right, nginx was limited in regards to sticky sessions.

Unfortunately I am running Windows. I probably could stand up a new Linux box and put HAproxy on it but it would involve considerable DNS re-arranging.

Mark,

Did Tim’s example file help you? Tim shared his with me and it’s rocking on Amazon Lightsail linux machine. Maybe show us your config file?

I will try Tim’s example file on Tuesday (March 7). I have already incorporated most of the things in Tim’s config but there were a few items that might help.

One major difference is I am running on Windows.

Also I got NGINX working on one Windows server but not on another with essentially the same config EXCEPT the one that worked used sub domains.

I will give some feedback at the end of Tuesday.

Here is the latest FAILURE …

All I am trying to do is a simple proxy on a Windows server behind a network firewall (not server firewall) that does not allow any ports except 80 and 443. The stand alone app works fine when inside the network firewall when you use the URL with the port used in the stand alone compile (no proxy). I can open from inside the network firewall without using the NGINX proxy on the correct IP or on the server itself with 127.0.0.1 and the correct port used in the compile (8081 and others I have tried). The Windows firewall ports I am testing are open. I even tested with the Windows firewall off on all zones. If the ports are not open the stand alone app will not launch from the LAN side of the network and just gives a “Site unavailable” message.

With NGIN running and configured as a simple proxy I get the famous “The application has gone offline” message but it is not formatted correctly.

If the image does not display here is a link to download it.
https://www.dropbox.com/s/extcqt3aszbyu6w/NGINX_ProxyError.jpg?dl=0

I incorporated just about everything in Tim’s example config in various combinations and permutations. One of two things happens. I ether get a “Site Cannot Be Reached” or the above error with a mis formatted “Application Has Gone Off Line” error from the Xojo app itself.

In my testing I have tried hard coded IP addresses and domain names that resolve to an IP on the server. I have also use 127.0.0.1 as the proxy_pass target parameter in the NGINX config.

The strange part is this almost identical config but a different stand alone program (same compiler settings) works just fine on a different server. Both servers have more than one IP address. The problem server has been rebooted.

So … What would cause the mis formatted “The application has gone off line” message?

Hey Mark,

Here’s my nginx file, based on Tim Dietrich’s file, that is working with two apps that are accessed via “appname1.foo.com” and “appname2.foo.com”. This works on Linux running on a Lightsail server, but I’m not sure how it may differ from Windows…

# NGINX config file.
# To reload: sudo nginx -s reload

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;

events {
worker_connections 1024;
}

http {

# ######################################################

# Disables the default host.
# See: http://serverfault.com/questions/420351/best-way-to-prevent-default-server
server { 
listen 80 default_server;
server_name _; 
return 444; 
}

# ######################################################

server {
listen 80;
server_name appname1.foo.com;
location '/.well-known/acme-challenge' { default_type "text/plain"; root /var/www/html; }
location / { return 301 https://$server_name$request_uri;  }
}

server {
listen 443 ssl;
server_name appname1.foo.com;
ssl_certificate /etc/letsencrypt/live/appname1.foo.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/appname1.foo.com/privkey.pem;
location / { proxy_pass http://127.0.0.1:1111; }
}

# ######################################################

server {
listen 80;
server_name appname2.foo.com;
location '/.well-known/acme-challenge' { default_type "text/plain"; root /var/www/html; }
location / { return 301 https://$server_name$request_uri;  }
}

server {
listen 443 ssl;
server_name appname2.foo.com;
ssl_certificate /etc/letsencrypt/live/appname2.foo.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/appname2.foo.com/privkey.pem;
location / { proxy_pass http://127.0.0.1:2222; }
}

# ######################################################

}

You’ll notice that each app is setup exactly the same, other than the app name…

This allows Let’s Encrypt to verify the certificate during the certicate creation process:

location '/.well-known/acme-challenge' { default_type "text/plain"; root /var/www/html; }

This redirects the user from port 80 to 443 when trying to access the app.

location / { return 301 https://$server_name$request_uri;  }

This uses the Let’s Encrypt SSL certficate.

ssl_certificate /etc/letsencrypt/live/appname2.foo.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/appname2.foo.com/privkey.pem;

Hal,

I have done everything you have in your config and I have also tried without doing any HTTPS and just a simple HTTP proxy from one port to another.

I have even compiled the app as a Windows 32 bit and 64 bit and both give the same “The application has gone off line” message. The app starts then bails with this message and it is formatted badly.

If you skip the proxy and go direct to the port that is compiled in the app it works just fine … so long as I am behind the network firewall.

So … I am stuck.

Mark

That’s a bummer. Sorry that didn’t help.

Hal,

Thanks for trying. Since I have this working on another Windows Server (same OS) I know it can work. I may have something else wrong and un-related on the server that makes this happen.

Both are Virtual Machines but one is on a public cloud service and the other on a piece of hardware running VMWare ESXi at a university (the one that does not work).

If I figure it out I will post the details here for others.

Thanks, Mark

On the ESXi VM you can open http://127.0.0.1:PORT in your browser and the app comes up and works?

Phillip,

From the VM where the app is hosted … Yes http://127.0.0.1:PORT works.

It also works from another computer on the LAN behind the network firewall with http://:PORT

BUT if I try to use the “front end” IP defined in the proxy via NGINX I get the mis formatted “Your application has gone off line”

Mark,

Can you post your actual config - I was referencing one above but that doesn’t look to be what you are using.

If you can open up your app with http://:PORT from your LAN then what are you trying to achieve here?

Browsers default to port 80 so if you try to map an IP address to a Xojo app then you have to redirect port 80->9000 for example and you can only use one app on the default port 80. On a LAN there is no point in doing that as you can just bookmark port 9000 or link to it from an Intranet.

Now when you say:

[quote=319372:@Mark Strickland]

BUT if I try to use the “front end” IP defined in the proxy via NGINX I get the mis formatted “Your application has gone off line”[/quote]

That is very interesting because I don’t understand why you would use a front end IP at all. Reverse proxies are mostly used for virtual host hosting meaning you want to run more than one app on port 80 differentiated by virtual hosts. Nginx can only listen on IP addresses mapped to that VM which means you could just as easily do http://:PORT without Nginx which you have indicated works. So I am going to assume you meant a domain or subdomain of some kind.

You want:

http://myapp.mycompany.tld -> http://:

Yay/nay?

Last but not least the fact that you get “Your application has gone offline” means the initial request to the Xojo web app succeeded. The browser downloaded the javascript and started to execute it. However somewhere between the browser receiving the initial response and sending subsequent queries (the javascript framework) it stopped getting responses and assumed the app went offline.

This is most indicative of the Xojo web app actually crashing. You insist it’s still available however at this time (not crashed).

Have you looked into the Nginx error log to see if it gives any clues? Have you disabled Windows firewall just to rule it out?

Dummy virtual path is the key here…

[quote=318778:@Mark Strickland]
Oddly I have, on a totally different Windows Server 2008 r2 box, gotten this to work using Sub-Domains to route to the proper app all on the same public IP. I have just about duplicated the config from this server and I just get the badly formatted “gone off-line” message.[/quote]

So the two boxes are NOT the same then…

These two ideas are VERY different.

subdomain1.domain.tld → http://127.0.0.1:9000 (WILL WORK)
subdomain2.domain.tld → http://127.0.0.1:9001 (WILL WORK)

subdomain.domain.tld/app1 → http://127.0.0.1:9000 (WILL NOT WORK)
subdomain.domain.tld/app2 → http://127.0.0.1:9001 (WILL NOT WORK)

Why will it not work? Simple… same reason https://forum.xojo.com/phillipscoolfolder does not work. The underlying server here (Xojo Web App) has no idea what that is.

You can configure a reverse proxy to forward a “dummy virtual path” as you describe however you have to do some URL rewriting to REMOVE that “dummy virtual path” from the request that ultimately arrives at the Xojo web app.

Phillip,

The goal is to create a “front end” that can accept connections over port 443 and route the to one of several stand alone Xojo apps on different port on the back end.

The network firewall is routinely only open to 80 and 443 BUT they will allow other ports on request. When they did a network scan to the port (8443) currently running the stand alone Xojo app they said they found some “weak ciphers” (example: TLS_RSA_WITH_3DES_EDE_CBC_SHA) and they would NOT open the port without me disabling those ciphers on the app. In a stand alone Xojo app I don’t see any settings to disable or enable any ciphers. I am assuming their socket library “bakes in” lots of ciphers.

So my theory was I could use a path (trailing part of the URL) and have the proxy intercept the inbound URL and redirect it to the proper internal port based on the the path and strip off the path. All of this would avoid opening these ports to the outside and the issue of the “weak” ciphers.

So far all of my testing does NOT involve a path. I just wanted to get the port remapped. ALSO all of my testing was to a “dummy” port (not 80 or 443) but inside of the network firewall to test the proxy theory by just redirecting a port then add on the path later.

I easily got this working on a different Windows server with a subdomain (no a path).

The problem server does have three sub-domains setup. I am currently using Abyss and CGI (too slow). I took down Abyss and tried to use one of the sub-domains to do the proxy on port 80 but still the same problem.

My hope was to figure out the NGINX config then simply take down Abyss permanently and substitute NGINX and all stand alone Xojo apps. Since this is a production environment I could not just shut down the apps until I got everything working.

AND YES … I have disabled the Windows firewall but no difference.

I will post the config tomorrow.

Thanks for the help.

Mark

If it were me I would stick with Abyss which is one of the best web servers available on Windows outside of IIS.

Abyss supports reverse proxying so if you know it works stick with that and skip Nginx.