Catching HttpSocket redirect final URL

General
1

I am using code based on the Xojo downloader example, fixed to auto use the file url etc.

When my HTMLviewer detects a file download it downloads the file using httpsocket and it works great, all but when the file is a redirected download. Example…

www.site1.com/file.zip

redirects to

www.site2.com/folder/file.zip

This is being done via a 302 redirect and I am detecting this in the HeadersReceived event on the httpsocket

What I want to do is catch the final “true” url of the file and have the socket download it from the redirected url

2

In HTTP 30X responses, the redirect URL is included as the Location header in the HeadersReceived event:

Sub HeadersReceived(headers as internetHeaders, httpStatus as integer) Dim url As String = headers.CommaSeparatedValues("Location") Me.Get(url) End Sub

3

Thanks Andrew!

[quote=331626:@Andrew Lambert]In HTTP 30X responses, the redirect URL is included as the Location header in the HeadersReceived event:

Sub HeadersReceived(headers as internetHeaders, httpStatus as integer) Dim url As String = headers.CommaSeparatedValues("Location") Me.Get(url) End Sub [/quote]

4

Any code available to show how this needs to be handled? I’m trying to do the same, but I get stuck on the different 30X responses like the “Moved permanently” one.
Or does anyone know if there are plugins (Win/Mac) that handle this? Never thought that saving the data you see with “View Source” is so hard to save on your hard disk…

5

This thread pretty much explains itself.
Did you try the suggestion above? What happened when you did?

Alternatively, the new framework socket follows redirects automatically.

6

I used this but I get stuck on https://daardaar.be/rubriques/culture-et-medias/johnny-hallyday-ce-belge-snobe-en-flandre/

Sub HeadersReceived(headers as internetHeaders, httpStatus as integer) Handles HeadersReceived Dim loopCount As Integer = 0 While (httpStatus = 301 or httpStatus = 302) TextAreaHeaders.Text=headers.Source Dim url As String = headers.CommaSeparatedValues("Location") DownloaderSource.Get(url) loopCount = loopCount + 1 If (loopCount > 10) Then System.DebugLog("Too many redirects.") Return End If Wend End Sub

I get this in my TextArea:

HTTP/1.1 301 Moved Permanently Date: Mon, 11 Dec 2017 16:48:36 GMT Content-Type: text/html Content-Length: 178 Location: https://daardaar.be/rubriques/culture-et-medias/johnny-hallyday-ce-belge-snobe-en-flandre/ X-FW-Hash: 3djkfwj6mw Server: Flywheel/4.1.0 X-FW-Serve: TRUE X-Cache: HIT X-Hits: 221 X-FW-Static: NO X-FW-Type: VISIT Connection: close

7

Are you using a synchronous .Get call? You would need to subclass HTTPSocket to handle the HeadersRecieved and follow the redirect to get the actual content.

8

Here’s a little example project using a new framework socket: https://xojo.io/httpsocket-101
That should handle a 301 and 302 automatically for you. I did not include error handling.

9

Thanks Tim. I’m now looking into the new framework socket. I didn’t know it was taking care of the redirects.

10

Handles SSL stuff for you too. The new framework socket is a huge improvement.

11

Spent hours on this – thanks to your help // thanks to you // my code is now working and 100 times better then I imagined. Thank you Tim, really appreciated!

12

[quote=364283:@Gert Van Assche]I used this but I get stuck on https://daardaar.be/rubriques/culture-et-medias/johnny-hallyday-ce-belge-snobe-en-flandre/
…

HTTP/1.1 301 Moved Permanently Date: Mon, 11 Dec 2017 16:48:36 GMT Content-Type: text/html Content-Length: 178 Location: https://daardaar.be/rubriques/culture-et-medias/johnny-hallyday-ce-belge-snobe-en-flandre/ X-FW-Hash: 3djkfwj6mw Server: Flywheel/4.1.0 X-FW-Serve: TRUE X-Cache: HIT X-Hits: 221 X-FW-Static: NO X-FW-Type: VISIT Connection: close[/quote]

It seems the URL you quote is moved permanently to itself. Is there a reason this is done, or am I just confused as usual. Can’t see any cookies.

13

the link does not work anymore. does anyone have the example project?

14

I’ve fixed the DNS settings for Xojo.io. You should be able to access the files again as the change propagates.

15

thanks tim… but i’m still not able to get the final redirect url. i don’t know what to do with the txtResult.text content

16

hi @Gert Van Assche how did you solve this?

17

The example project used Xojo.Net.HTTPSocket - the old version of URLConnection. If URLConnection is not yielding the results you want, you may be facing an intentional block to prevent you from downloading the file. Dropbox is tricky like that.