Help with HTTPSocket

I am working on an application used by Amateur (Ham) Radio Operators. The program will access a web site (QRZ.com) to first login the user and later retrieve information about other radio operators. The website offers an XML account service that I am trying to access via an HTTPSocket.

I have been unable to get a response using the HTTPSocket, but an HTMLViewer displays the XML response without a problem.

I have a Main Window with a push button and an HTMLViewer. The following code is contained in the Push Button Action event handler.

Dim qrz_socket As New HTTPSocket
Dim qrz_response As String
Dim qrz_query As String

qrz_query = “http://xmldata.qrz.com/xml/1.27/?username=WN2VNF
HTMLViewer1.LoadURL( qrz_query )
qrz_response = qrz_socket.Get( qrz_query, 10 )

Return

Running the program results in nothing being returned from the HTTPSocket.Get call, but the HTMLViewer display the following information received:

<?xml version="1.0" encoding="iso8859-1" ?>
  • Username / password required Fri Dec 6 14:49:58 2013 cpu: 0.055s

I assume I am missing something simple but have not yet figured it out.

Please help !

Thanks in advance.

I just pasted your code above and I get the this XML returned back in qrz_response:

<?xml version="1.0" encoding="iso8859-1" ?> <QRZDatabase version="1.27" xmlns="http://xmldata.qrz.com"> <Session> <Error>Username / password required</Error> <GMTime>Fri Dec 6 16:16:32 2013</GMTime> <Remark>cpu: 0.026s</Remark> </Session> </QRZDatabase>

I added one line:

TextArea1.Text = DefineEncoding(qrz_response, Encodings.UTF8)

Paul - I must be missing something very simple. Here’s my complete code in the Push Button Action event handler:

Dim qrz_socket As New HTTPSocket
Dim qrz_response As String
Dim qrz_query As String

qrz_query = "http://xmldata.qrz.com/xml/1.27/?username=WN2VNF"
HTMLViewer1.LoadURL( qrz_query )
qrz_response = qrz_socket.Get( qrz_query, 10 )
MsgBox( DefineEncoding(qrz_response, Encodings.UTF8) )

Again - The socket.Get command hangs for the 10 second timeout and nothing shows in the message box.

However, the HTML Viewer displays the response just fine.

I get the same result as Paul. No delay, immediate response of “username / password required” xml message.

Very interesting… Something is causing the firewall (AVG Internet Security) to block the HTTPSocket.Get but does not block the HTMLViewer.LoadURL

How would I go about finding out what is different between the call made by HTMLViewer.LoadURL and the call made by HTTPSocket.Get ?

Ron Bower
Ellicott City, MD

http://www.wireshark.org

Hi Ron.

I ran into your post after having similar problems. I tried your code and it does the same for me as you. AVG doesn’t seam to be the cause, nor does windows firewall.

I’ve been running some tests back and forth with my server changing headers using php and found something interesting:

If i send headers similar to this:
Content-Length: 475
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive

It works, but if I send headers similar to what your http://xmldata.qrz.com/xml/1.27/?username=WN2VNF sends:
Connection: close

It does not work, and I see no content length in the html source viewer (i’m using an old OstroSoft Internet Tools app to look at it)

I can’t even get www.google.com, they send no content-length either. However, they do on 404 pages so I tried www.google.com/k and it worked.

Is this a bug in HTTPSocket? or am I missing a setting for it?

Sorry I don’t have any answer for you, but I did get closer to the cause. Hopefully this will spark an idea from another member, as I’m also hoping for a solution.

Scott - Thank you for the information. The HTTP and IP protocols are totally foreign to me, though.

It would be nice if someone from Xojo could tell us how HTTPSocket.Get is coded differently than the HTMLViewer.LoadURL function

Ron Bower

HTTPSocket is rather Raw. It just asks the server for the data at the URL you provide. HTMLViewer has webkit behind it, so it’ll run any javascript and interpret any headers that are delivered (including redirects).

Looking at your examples, it appears that the main issue is that HTTPSocket is using the 1.0 protocol and the server you are connecting to is expecting and responding in HTTP/1.1. (Connection: Keep-Alive and Connection: Close are both part of the 1.1 protocol)

Greg - Thanks for that information.

So the question (problem) for me is quite simple… If I can;t use HTTPSocket.Get to retrieve the information form the URL, what can I use ?

Thanks in advance.

Ron Bower
Ellicott City, MD

Before you do that, try setting the user-agent header for the httpsocket. It could be that they only return content for something that says it is a browser. You should be able to get a valid user agent string by doing a google search.

Okay, Greg - I’ve got some studying to do. Thanks for the help and suggestion.