HTTPSocket, Apache and duplicate Host headers

I have an old app that is in the field. It uses a simple HTTPSocket “GET” request to the current version number of the software, and if it’s old, it puts up a dialog box reminding the user to update. The server it connects to is running Apache, and has a little CGI script which returns the current version number. This has worked fine for years.

Recently, my hosting company did a bunch of software updates, which included an Apache update. Since then, the server is returning 400 responses (“Bad Parameters”). However, this 400 response only happens when running the Xojo app using HTTPSocket. If I use CURL or simply a web browser, the server returns the proper 200 response with data. This works even when I force CURL to use the old HTTP/1.0 request style.

Digging deeper, I captured packets on my mac ( see https://support.apple.com/en-us/HT202013 - thanks Apple! ) and determined the problem - in my Xojo app I was manually adding the Host header before the HTTPSocket request but apparently when you do that, you end up with two host headers in the request:

	GET /version.cgi HTTP/1.0
	Accept: */*
	Accept-Language: en
	Host: mycompany.com
	Host: mycompany.com

This duplicated Host: header field seems to trigger the latest version of Apache to give a 400 (bad parameters) response.

This is on a shared server, so my ability to do some things (e.g. revert the Apache version) is limited on the server side. The app is in the field, so I can’t update it to fix the bug on the client side.

Any ideas?

Why are you manually adding the host header?

History, I think.

Note that this is a HTTP/1.0 request - and HTTP/1.0 didn’t originally have Host headers.

I think what happened was

  • in older versions of REALbasic, HTTPSocket didn’t add the host header
  • so I added it manually
  • then REALbasic or Xojo got updated, and started putting the host header in automatically
  • but I didn’t know to remove it from my code

Since you can’t change your old app, you have to do something on server-side…
Maybe you can edit the apache configuration for your virtual host on the shared server?
Apache Module mod_headers comes to mind…

Not sure you can resolve this on your own. Apache is blowing it up before you’ll be able to modify the request.

Hopefully the url is a subdomain of your site?

Thanks for the ideas:

I tried this in my .htaccess file:

# remove all Host headers (which might remove duplicate one)?
RequestHeader unset Host:
# set back the correct one (not sure if this is necessary)
RequestHeader set Host: "mycompany.com"
# set a header just to see if mod_headers is working at all:
Header set TestHeader: "foobar"

but I see no change, still 400 errors.

Edit: I also tried the early keyword, no difference.

.htaccess only gets processed after Apache has validated the request, identified which virtual server, which folder, etc. Apache is outright rejecting the request. You never get as far to look at the .htaccess.

This seems to be the case. And this is not a subdomain, the URL is basically mycompany.com/version.cgi

How / why would a subdomain help?

@Michael Diehr subdomain would help because you could set up another server somewhere that doesn’t blow up on multiple host headers and serve the update script there.

In this case I am not sure there are any good options.

not being an Apache specialist… would the same happen if this would be configured in the vhost-configuration (rather than .htaccess)? or might a mod_rewrite or mod_headers in the vhost.config work?

I couldn’t say without testing a vanilla Apache whether it’s rejecting it wholesale or if its some module the host has installed to prevent bad requests. It is worth a try if they have access to change their vhost config.

I recently ran into the same scenario - working fine for years then not.

Here’s what I found in my case - maybe it can be helpful in yours:

As I built up my URL, I was running EncodeBase64 on one of the parameters and neglected to set linewrap to 0. Thus the url contained linefeeds. I tested with this setup when I developed it and it always worked fine, until recently.

Apparently earlier Apache versions were not enforcing RFC 7230 §3.2 fully and were tolerant of linefeeds - but the latest version is no longer tolerant - Apache refuses the request.

If you problem is along these lines and you have access to the server config file - you can include this directive and revert Apache to previous behavior:

HttpProtocolOptions Unsafe

More information is here:

https://httpd.apache.org/docs/2.4/mod/core.html

I too think this might be a solution. Will check with the hosting company to see if they allow it.

Update: I was never able to come up with an easy server-side solution, so I ended up doing a .0.1 update of my old app which stopped adding the duplicate header.

This doesn’t solve the problem of old customers no longer being able to update, but solves the issue for new customers who want to use the old version.