Handling web-authenticated proxies - Ideas?

Hi.

Currently, HTTPSocket is able to manage both unauthenticated proxy connections and http-auth authenticated ones.

Problem is it’s increasingly common to see proxies that require a web-form to be filled for authentication. In the case of my company this web form is a full-on html form with a captcha and all.

Has anyone faced something like this? How to detect this and provide the proper htmlviewer page?

Also: Is there a simple way to pull the proxy data from the system in both OSX and Windows?

The solution is to talk to IT and have this proxy removed.

Or show in htmlviewer and let the user do it.

[quote=124943:@Christian Schmitz]The solution is to talk to IT and have this proxy removed.

Or show in htmlviewer and let the user do it.[/quote]

It’s fair to assume that a 305,000 employees (and twice as many outsourced implants) company who just recently completed a huge project to implement a proxy/caching service that finally complies with “Fed” policy won’t be open to turning the whole thing off because I don’t know how to implement it in Xojo :smiley:

I know I should show it in HTMLViewer. I said as much. I was wondering how had people done it, if anyone had ever faced it. Like how smartphones will show the webpage for WiFi authentication, when do they know to do so?

(BTW: I’m “IT” in this context, but the level at which this proxy is set-up, I can’t avoid it. It’s a worldwide distributed triple-secured, SOx-Controlled proxy/caching solution).

And that all from only one dictionary, Wow! :smiley:

Seems you are not alone facing that problem. SO has several threads about that very issue :
http://stackoverflow.com/search?q=wifi+web+authentication
http://stackoverflow.com/search?q=captive+portal

Dave started a thread about a similar subject a while ago :
https://forum.xojo.com/6012-send-data-to-webpage

You participated in https://forum.xojo.com/10694-web-robot with a suggestion for http://phantomjs.org/
Would that not be a solution ?

Maybe simply going to the login URL and sending the required information ?

Copied and pasted from their notification email, too.

[quote=124983:@Michel Bujardet]Seems you are not alone facing that problem. SO has several threads about that very issue :
http://stackoverflow.com/search?q=wifi+web+authentication
http://stackoverflow.com/search?q=captive+portal

Dave started a thread about a similar subject a while ago :
https://forum.xojo.com/6012-send-data-to-webpage

You participated in https://forum.xojo.com/10694-web-robot with a suggestion for http://phantomjs.org/
Would that not be a solution ?

Maybe simply going to the login URL and sending the required information ?[/quote]

Yeah. The issue is not automating the data entry (which would be impossible, it has a captcha that requires you to manually manipulate some images according to an instruction).

It’s that the proxy doesn’t show always, but in some pages or after some indeterminate (and probably random to avoid prediction) amount of time. It’s rules are as inscrutable as the ones Esotalk uses to decide you need to log in back again :smiley:

So the idea is that if the proxy requires authentication that is shown, and if not then the results are shown. The problem is knowing when there is authentication required and when not.

And I’m not even going into how many redirects happen from first query to final display, but I’ve counted eight.

[quote=124983:@Michel Bujardet]Seems you are not alone facing that problem. SO has several threads about that very issue :
http://stackoverflow.com/search?q=wifi+web+authentication
http://stackoverflow.com/search?q=captive+portal

Dave started a thread about a similar subject a while ago :
https://forum.xojo.com/6012-send-data-to-webpage

You participated in https://forum.xojo.com/10694-web-robot with a suggestion for http://phantomjs.org/
Would that not be a solution ?

Maybe simply going to the login URL and sending the required information ?[/quote]

I didn’t comment on PhantomJS. Sorry: It works for automation and it’s pretty good. But I don’t need to automate, I need to discern when user input is required. The form from the proxy may be indistinguishable from the target URL, other than not showing the expected results.

What about looking at the DOM for patterns and controls pertaining to the login page ? A bit like browsers spot ‘username’ and ‘password’ fields ?

Thanks to @Michel Bujardet I now know this is called a “captive portal”. I wasn’t aware of this, and it applies both to the set-up our proxy uses and to authenticated WiFis like those used in airports and hotels, which require pre-payment.

Even though this is common, the solution seems to be pretty low-tech: Attempt to connect to a webpage you know will always be the same:

http://www.apple.com/library/test/success.html
http://www.msftncsi.com/
http://clients3.google.com/generate_204
http://www.google.com/blank.html

(for Wi-Fi it’s also suggested to first try a DNS query to Apple, as some authentication may not even give proper DHCP data before authentication).

If the result is not as expected, follow the redirects and display the last in an htmlview if it’s valid HTML or present an error to the user that the destination could not be reached (and hint at proxies or firewalls). Some of these, if the user has authenticated in a web browser, will work in apps as they might be IP-based.

For WiFi there’s now a protocol called WISPr, which in reality uses the above but includes special metadata with the details:

http://msdn.microsoft.com/en-us/library/windows/hardware/dn408679.aspx

This doesn’t apply to me in this scenario, but for completeness case.

Thanks, michel!