NGINX - Proxy making me crazy

  1. ‹ Older
  2. 3 years ago

    Mark S

    7 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    Phillip,

    From the VM where the app is hosted ... Yes http://127.0.0.1:PORT works.

    It also works from another computer on the LAN behind the network firewall with http://<ip of server>:PORT

    BUT if I try to use the "front end" IP defined in the proxy via NGINX I get the mis formatted "Your application has gone off line"

  3. Phillip Z

    7 Mar 2017 Pre-Release Testers, Xojo Pro Florence, SC
    Edited 3 years ago

    Mark,

    Can you post your actual config - I was referencing one above but that doesn't look to be what you are using.

    If you can open up your app with http://<ip of server>:PORT from your LAN then what are you trying to achieve here?

    Browsers default to port 80 so if you try to map an IP address to a Xojo app then you have to redirect port 80->9000 for example and you can only use one app on the default port 80. On a LAN there is no point in doing that as you can just bookmark port 9000 or link to it from an Intranet.

    Now when you say:

    @Mark S
    BUT if I try to use the "front end" IP defined in the proxy via NGINX I get the mis formatted "Your application has gone off line"

    That is very interesting because I don't understand why you would use a front end IP at all. Reverse proxies are mostly used for virtual host hosting meaning you want to run more than one app on port 80 differentiated by virtual hosts. Nginx can only listen on IP addresses mapped to that VM which means you could just as easily do http://<ip of server>:PORT without Nginx which you have indicated works. So I am going to assume you meant a domain or subdomain of some kind.

    You want:

    http://myapp.mycompany.tld -> http://<ip of server>:<PORT>

    Yay/nay?

    Last but not least the fact that you get "Your application has gone offline" means the initial request to the Xojo web app succeeded. The browser downloaded the javascript and started to execute it. However somewhere between the browser receiving the initial response and sending subsequent queries (the javascript framework) it stopped getting responses and assumed the app went offline.

    This is most indicative of the Xojo web app actually crashing. You insist it's still available however at this time (not crashed).

    Have you looked into the Nginx error log to see if it gives any clues? Have you disabled Windows firewall just to rule it out?

  4. Phillip Z

    7 Mar 2017 Pre-Release Testers, Xojo Pro Florence, SC

    @Mark S My goal is to use a single domain name on port 443 (HTTPS) to launch several backend stand alone apps, adding a dummy virtual path at the end of the URL. I cannot use port numbers directly because I am behind a network firewall and I don't have any control of the settings.

    Dummy virtual path is the key here...

    @Mark S Oddly I have, on a totally different Windows Server 2008 r2 box, gotten this to work using Sub-Domains to route to the proper app all on the same public IP. I have just about duplicated the config from this server and I just get the badly formatted "gone off-line" message.

    So the two boxes are NOT the same then...

    These two ideas are VERY different.

    subdomain1.domain.tld -> http://127.0.0.1:9000 (WILL WORK)
    subdomain2.domain.tld -> http://127.0.0.1:9001 (WILL WORK)

    subdomain.domain.tld/app1 -> http://127.0.0.1:9000 (WILL NOT WORK)
    subdomain.domain.tld/app2 -> http://127.0.0.1:9001 (WILL NOT WORK)

    Why will it not work? Simple... same reason https://forum.xojo.com/phillipscoolfolder does not work. The underlying server here (Xojo Web App) has no idea what that is.

    You can configure a reverse proxy to forward a "dummy virtual path" as you describe however you have to do some URL rewriting to REMOVE that "dummy virtual path" from the request that ultimately arrives at the Xojo web app.

  5. Mark S

    7 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    Phillip,

    The goal is to create a "front end" that can accept connections over port 443 and route the to one of several stand alone Xojo apps on different port on the back end.

    The network firewall is routinely only open to 80 and 443 BUT they will allow other ports on request. When they did a network scan to the port (8443) currently running the stand alone Xojo app they said they found some "weak ciphers" (example: TLS_RSA_WITH_3DES_EDE_CBC_SHA) and they would NOT open the port without me disabling those ciphers on the app. In a stand alone Xojo app I don't see any settings to disable or enable any ciphers. I am assuming their socket library "bakes in" lots of ciphers.

    So my theory was I could use a path (trailing part of the URL) and have the proxy intercept the inbound URL and redirect it to the proper internal port based on the the path and strip off the path. All of this would avoid opening these ports to the outside and the issue of the "weak" ciphers.

    So far all of my testing does NOT involve a path. I just wanted to get the port remapped. ALSO all of my testing was to a "dummy" port (not 80 or 443) but inside of the network firewall to test the proxy theory by just redirecting a port then add on the path later.

    I easily got this working on a different Windows server with a subdomain (no a path).

    The problem server does have three sub-domains setup. I am currently using Abyss and CGI (too slow). I took down Abyss and tried to use one of the sub-domains to do the proxy on port 80 but still the same problem.

    My hope was to figure out the NGINX config then simply take down Abyss permanently and substitute NGINX and all stand alone Xojo apps. Since this is a production environment I could not just shut down the apps until I got everything working.

    AND YES ... I have disabled the Windows firewall but no difference.

    I will post the config tomorrow.

    Thanks for the help.

    Mark

  6. Phillip Z

    7 Mar 2017 Pre-Release Testers, Xojo Pro Florence, SC

    If it were me I would stick with Abyss which is one of the best web servers available on Windows outside of IIS.

    Abyss supports reverse proxying so if you know it works stick with that and skip Nginx.

  7. Mark S

    9 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    Phillip,

    Yes I like Abyss but I could not get it to work either. It had the same symptom of a mis-formatted "Gone Off Line" message so I assumed that how Abyss handled things was different. That is why I started looking at NGINX.

    Now I am thinking that maybe the server itself is the issue.

    But .... I am at a loss on how to debug at this point because I have tried everything I can think of.

    Mark

  8. Mark S

    11 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    Good progress but different server. ONE PROBLEM REMAINS

    I managed to get NGINX working on a different Windows server 2008 R2. This server is very similar to the other one but works with Tim's config. (I will return to the other server later.)

    The only problem I now have is there are two domains (actually sub-domains) and they run different Stand Alone Xojo apps on different ports. NGINX is handling the certs up front and they are proxied to different ports like this : 127.0.0.1:<port>. It works fine if I go directly to the HTTPS URL.

    I wanted to have NGINX do an HTTP 301 and force HTTP to HTTPS.

    Here is the NGINX to redirect to the HTTPS site from HTTP:

        #This forces http to https
        server {
            listen 80;
            server_name <sub-domain-1>.sbswebapp.com <sub-domain-2>.sbswebapp.com;
            return 301 https://$server_name$request_uri;
        }

    This redirect does work but it always ends up at the same sub-domain no matter which sub-domain is used in the URL in the browser.

    I would have thought that the RETURN 301 line with the parameters would do the correct redirect.

    It seems to be possibly browser cache related. If I clear the cache then it always goes to the first sub-domain I use after clearing no matter which one I specify in the URL.

    I have these in the NGINX config (in the location section) but it does not help:

    proxy_buffering off;
    proxy_cache off;

    My temporary work around is simply deny HTTP (no S).

    Any thoughts on an HTTP 301 redirect would be appreciated.

    Thanks.

  9. Mark S

    11 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    OK .... SOLVED

    I created TWO of these blocks and hard coded the redirect:

     #This forces http to https
    
    #Xojo App 1
        server {
            listen 80;
            server_name <sub-domain-1>.sbswebapp.com
            return 301 https://<sub-domain-1>.sbswebapp.com
        }
     
    #Xojo App 2
        server {
            listen 80;
            server_name <sub-domain-2>.sbswebapp.com;
            return 301 https://<sub-domain-2>.sbswebapp.com
        }

    Hard coding the server name without a URI was OK because I don't have any URI's in either of these apps.

    The browser cache did require clearing but after that each sub-domain HTTP did the correct redirection to HTTPS.

    SO ... the browser cache does play a part some how. Can the NGINX server force the browser to NOT follow the cache? Not sure.

  10. Mark S

    11 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    I am posting the details of what I have learned to maybe help others. It appears that 301 redirects are sometimes cached by different browsers.

    Here is an article on which browsers cache 301 redirects:
    http://stackoverflow.com/questions/9130422/how-long-do-browsers-cache-http-301s#21396547

    Here is a Wikipedia page on all of the different codes. Note that there are many 30x versions.
    https://en.wikipedia.org/wiki/List_of_HTTP_status_codes

  11. Mark S

    13 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    One more update then I am done.

    I went back to my original server that was giving problems. I made sure the config was similar to the working Windows server. I cleared the cache in my browser ..... AND SUCCESS.

    Apparently the cache in ALL of my browsers were pointing somehow to the site that returned an immediate "This Application Has Gone Off Line". I suppose as I was testing I tried the same "broken" NGINX config with three different browsers and it was cached on all three.

    Ahhhhh ..... how software tries to "silently help" and ends up doing the wrong thing.

    SO .... The net result:

    1) NGINX for Windows CAN be a front end for Xojo Stand Alone Apps using the proxy_pass parameter.

    2) NGINX can easily handle SSL certificates then connect to a NON-SSL HTTP Stand Alone App on a non-standard port of your choice while behind the Windows firewall on that server that does not have the non-standard port open to the public. Essentially you manage the backend Stand alone apps on IP 127.0.0.1:<various_ports>.
    The certs are VERY easy. Two config lines: One for the cert bundle and one for the private key

    3) NGINX can redirect a PUBLIC HTTP request to a PUBLIC HTTPS request to always force SSL.

    4) NGINX can load balance between several Xojo Stand Alone Apps. Although not throughly tested this appears to work.

    5) NGINX appears to be able to handle multiple domain names on the SAME IP and proxy them to the correct Stand Alone app (also not throughly tested).

    All of this seems to make NGINX a viable alternative to work as a "front end" to multiple Stand Alone Xojo apps to avoid the problems of CGI when performance is an issue. It also works in Windows and gives an alternative to HAProxy that is only available in Linux.

    I hope this helps somebody.

  12. Hal G

    13 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers CampSoftware.com

    Hey Mark,

    Could you post your NGINX config file that works for Windows? I'm not likely to run Web Apps on Windows, but it'd rock to have that option.

    In my config file that I posted in this thread for Linux, it includes what you mentioned for your items 1 - 4, although I'm using a Let's Encrypt cert that should automatically renew itself. I'll find out when it gets closer to the expiration date. I also haven't tested pointing to multiple app instances on the same box or different boxes yet...

  13. Mark S

    13 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    Here is based on my working config. I added lots of comments and made it sort of generic. Hopefully I did not introduce any typos.

    Enjoy.

    #This works in Windows Server.  Not tested in other OS's.
    
    #Note that forward slashes are used in file paths even in Windows.
    
    #Also this has been edited for this post so it is possible some syntax error might exist.
    
    #NOTE the <> characters in the domains and ports below should not be included in the config.
    #     Added for clarity.
    
    
    #Not used in Windows
    #user  nobody;
    
    worker_processes  1;
    
    #error_log  logs/error.log;
    #error_log  logs/error.log  notice;
    #error_log  logs/error.log  info;
    
    #pid        logs/nginx.pid;
    
    events {
        worker_connections  1024;
    }
    
    
    http {
        include       mime.types;
        default_type  application/octet-stream;
    
        server_names_hash_bucket_size 64;
    
        #You can change the log format if you want with something like this
        #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
        #                  '$status $body_bytes_sent "$http_referer" '
        #                  '"$http_user_agent" "$http_x_forwarded_for"';
    
        #access_log  logs/access.log  main;
    
        sendfile        on;
        #tcp_nopush     on;
    
        #keepalive_timeout  0;
        keepalive_timeout  65;
    
        gzip  on;
    
    
        #This is for load balancing.  See below for different proxy_pass directive.
        #If you wanted to load balance multiple different apps you would need more upstream
        #  directives with different names pointing to different ports for that group of apps.
        #This essentially gives a logical name to a group of server directives and attempts
        #  to distribute the load among the group.
        
        #Disabled for now.
        
        #upstream YourLogicalAppName {
        #    ip_hash;  #Needed for session persistence.
        #      My limited testing for ip_hash shows this may not result in evenly
        #        distributing the load among the different instances but Xojo must
        #        have session persistence.
        #    server 127.0.0.1:<XojoAppONEport1>;  #Instance One
        #    server 127.0.0.1:<XojoAppONEport2>;  #Instance Two
        #} #End Upstream
    
        #----------------------------------------
        #Normally you would do this return directive to force HTTP to HTTPS using variables ... but ...
        #  return 301 https://$server_name$request_uri;
        
        #I had trouble with this format when I had multiple domains resolving to the same IP.
        #My solution was to just "Hard Code" the return URL but other formats may work.
        
        #----------------------
        #This forces http to https for domain 1
        
        #Browser cache can cause problems with 301 redirects.
        #Read here for more:
        #http://stackoverflow.com/questions/9130422/how-long-do-browsers-cache-http-301s#21396547
        
        server {
            listen 80;
            server_name <YourDomainNameONE>;
            return 301 https://<YourDomainNameONE>;
        } #End Server
    
        #This forces http to https for domain 2 (if you need a second domain)
        server {
            listen 80;
            server_name <YourDomainNameTWO>;
            return 301 https://<YourDomainNameTWO>;
        } #End Server
    
    
    
    
        #----------------------
        server {
            server_name <YourDomainNameONE>;
            listen 443 ssl;
            #You can put your certs anywhere you want.
            #Note that in the bundle your site specific cert should be
            #  ADDED IN FRONT of the other intermediate certs if needed.
            #The server key should not have a password if you intend to run
            #  NGINX as a Windows service with NSSM.
            #  With a password you will be prompted when NGINX launches.
            #The password can be removed with OPENSSL.  Be careful not to expose the key file.
            ssl_certificate      C:/mycertdirectory/SBS_Bundle.crt;
            ssl_certificate_key  C:/mycertdirectory/server.key;
    
            location / {
                        proxy_buffering off;
                        proxy_cache off;
                        proxy_pass http://127.0.0.1:<XojoAppONEport>;
                        
                        #If you enable load balancing (upstream) above use this instead
                        #proxy_pass http:YourLogicalAppName;  #For load balancing
                        
                       } # End Location
    
        } #End Server
    
    
    
        #----------------------
        server {
            server_name <YourDomainNameTWO>;
            listen 443 ssl;
            ssl_certificate      C:/nginx-1.11.10/certs/SBS_Bundle.crt;
            ssl_certificate_key  C:/nginx-1.11.10/certs/server.key;
    
            location / {
                        proxy_buffering off;
                        proxy_cache off;
                        proxy_pass http://127.0.0.1:<XojoAppTWOport>;
                       } #End Location
    
        }# End Server
    
    } #end http
    
  14. Mark S

    19 Mar 2017 Pre-Release Testers, Xojo Pro, XDC Speakers

    I have discovered a few more settings are needed when you proxy to a stand alone app. Without these settings it appears the source of all sessions is 127.0.0.1. Adding these proxy_set_header parameters fixes that issue.

      location / {
        proxy_set_header  Host $host;
        proxy_set_header  X-Real-IP $remote_addr;
        proxy_set_header  X-Forwarded-Proto https;
        proxy_set_header  X-Forwarded-For $remote_addr;
        proxy_set_header  X-Forwarded-Host $remote_addr;
        proxy_pass        http://127.0.0.1:<your-port>;
      }
  15. last week

    Apologies for resurrecting a potentially dead thread, but has anyone gotten a definitive answer as to why this is an issue? I have absolutely no issue proxying other apps/frameworks, but Xojo is none too happy with the idea. I had it working earlier when I was directing the entire primary domain to the Xojo box. Currently I'm routing a subdomain as in the previous example:

    subdomain1.domain.tld -> http://127.0.0.1:9000 (WILL WORK)

    ...however, instead of 127.0.0.1, I'm routing to a private box on "10.x.x.x". As I mentioned, it worked before, but attempting to use a subdomain or an endpoint with a rewrite, it presents the "off line" error. Can anyone provide any further detail as to why Xojo specifically doesn't like this?

  16. John J

    Sep 10 Pre-Release Testers, Xojo Pro Cleveland, Ohio

    @MichaelWilliams - it is definitely doable, I have apps running behind proxies without any issue on subdomains. It is difficult to say exactly what the problem is from the info above, there are several unknown variables:

    If it worked before at 127.0.0.1:9000 but not does NOT work at 10.x.x.x then I would first of all check that the box with nginx on it has access to the server at 10.x.x.x:80/443 on the http or https port you are trying to reach - just to double check that firewalling or network routing is not the issue.

    If you can confirm that you can reach the standalone app without issue from the nginx box, then does your setup include any load balancing between standalone instances running on different ports? If so, not routing to the same instance consistently can cause the "offline" message and is fixed with sticky sessions (there are multiple ways to implement this) on the proxy.

    If it is not either of those things, are you routing requests from the root or a directory? (https://subdomain1.domain.tld/ or https://subdomain1.domain.tld/myapp/) The second option is more tricky to get working. I would advise against it unless absolutely necessary.

    Those are the first things that come to mind. LMK thx.

  17. Edited last week

    @John J -- The actual Xojo web instance is running on IIS on a Windows Server 2016. Nginx proxy is running on Ubuntu 18.04 box on the same internal network with a public facing IP. Yes, the Linux box has access to the Windows box over "10.x.x.x" -- we actually have other applications proxied and communicating on different ports just fine.

    Apologies for the vaguery, on the initial state. Previously, I'd redirected the primary domain "domain.tld" via "location /" and it worked. However, when reconfiguring the nginx instance, creating the same settings, but under "subdomain.domain.tld", I get the "This application has gone offline" error. Besides fiddling with the proxy header settings (even completely removing them) the only line of code I have is "proxy_pass http://10.x.x.x:9000" (same as before).

    Per Dev Tools, it can't retrieve the CSS or "framework/framework.js". I've tried adding new "location /framework" nonsense, as well as lines for static resources, but that just ends up going down a rabbit hole. At this point it's not completely obvious why it can't retrieve the resources.

    Any further insight you could provide would certainly be appreciated.

    NOTE: I'd already gone down the "/xojoendpoint" rabbit hole. Rewrites were required just to get to the normal failure of "location /", so it was no better.

  18. John J

    Sep 10 Pre-Release Testers, Xojo Pro Cleveland, Ohio

    Would you mind sharing the public IP if it is open to the internet? You can DM me if you prefer.

  19. John J

    Sep 10 Pre-Release Testers, Xojo Pro Cleveland, Ohio

    Actually the actual public hostname would be better

  20. Kevin W

    Sep 11 Pre-Release Testers, Xojo Pro

    I've run into something similar before, but I can't remember exactly what the issue was. I'm pretty sure there was already a discussion of it on these forums and a couple of feedback reports filed.

    I suspect that this issue will be resolved in the new framework, but it's just a guess.

    You might try searching feedback for web proxy and see what turns up.

  21. Edited last week

    Turns out that placing the following before the initial `location /` of the config file was sufficient:

    location ~ / {
        proxy_pass 10.x.x.x:port
    }

    I have no idea why a "case sensitive" flag would make any difference whatsoever, but everything loads now. Obviously more research will be necessary on my part as this makes zero sense, but it works. *shrug*

or Sign Up to reply!