2019 R1 Xojo.net.HTTPSocket and URLConnection

2019R1 has critital fixes for bugs in Xojo.Net.HTTPSocket and URLConnection:

54757 Crashes & Assertions » IDE Unhandled Exception NilObjectException no longer occurs in some situations when debugging Xojo.Net.HTTPSocket.
54412 Framework » All Xojo.Net.HTTPSocket events are no longer raised on a pre-emptive thread, this would cause all sorts of random/odd behavior depending on the thread context switches.
54502 Framework » All Xojo.Net.HTTPSocket HTTP POST now properly sends the request content on Windows.
54811 Framework » All URLConnection Send requests are now closed before the ContentReceived or FileReceived events are called.
54820 Framework » All URLConnection string encoding for the Response headers are now consistent across platforms. Also updated the string encoding for the the content string, returned in ContentReceived, to adopt the content-type encoding (charset).
54927 Framework » All URLConnection now automatically handles compressed content-encodings on Windows and Linux (by decompressing for you like macOS does).
54592 Framework » Windows URLConnection now correctly honors the timeout when SendSync is called (instead of always defaulting to 30 seconds)
54608 Framework » Windows URLConnection error messages are no longer empty on Windows.
54685 Framework » Windows URLConnection ReceivingProgressed event now correctly reports the totalBytes.
54822 Framework » Windows URLConnection response headers are now updated correctly each time a request is made, where before subsequent requests would include outdated headers.
54907 Framework » Windows URLConnection picks up proxy settings correctly.
54410 Framework » macOS URLConnection.SendSync no longer intermittently returns empty strings.

But, 2019R1 also has a bad bug that causes HTTPSockets and URLConnections to be very slow and use lots of CPU - see <https://xojo.com/issue/55301> . This bug is noted as being fixed for release in 2019R2. There’s a workaround in the feedback case but it seems to barely work in my testing (speeds are still 1/5 or lower of what they should be).

I really want to use 2019R1 now - has anyone found a way to do so? I wish we could get a 2019R1.1 release. Any idea how long until 2019R2?

Yes, that’s true - a really showstopper for us - we are still on 2018R3 because of this bug

+1

Here’s a summary of my testing so far:

2018R4 on mac : both URLConnection and Xojo.Net.HTTPSocket work fine.
2018R4 on Win : both URLConnection and Xojo.Net.HTTPSocket crash hard.

2019R1 on mac : seems to work fine, and with the thread workaround in https://xojo.com/issue/55301 I’m getting what appears to be nearly 100% speed.

2019R1 on Win : no more crashes, but even with the thread workaround, I’m only getting about 1/100th the speed (e.g. on a 300Mbps connection I’m getting 3Mbps)

Update: in 2019R1 on Windows, it turns out I was actually not doing the workaround as suggested here <https://xojo.com/issue/55301> : the workaround is to call .SendSync() inside a new thread, rather than .Send() on the main thread.

After making that change, it is faster (instead of 1% speed, I’m now seeing 10% speed) but still, a 90% slowdown is problematic.

However, it’s possible using the HTTP Range: header to download a byte range of a file, so another workaround is to use N downloaders to download the file in parallel, then re-assemble the pieces when done. I’m working on a sample class, and if I can get it to work I’ll add it to the bug report.

Yea, URLConnection in 2018R4 was a bad joke, where was the QA? But, after all the bug fixes in 2019r1 and since my app already had a .SendSync() inside a thread, URLConnection is working OK and fast wought for API calls. Are you using it for actuall file transfers?

Here’s a demo of a multi-socket downloader - it downloads a file using 1 to 10 separate URLConnections, and seems to be a pretty good workaround for the bug in 2019R1 for the time being.

Untested, needs error chekcing, use at your own risk, etc. etc:

https://xochi.com/xojo/urlconnection/urlconnection1.zip

The workaround of William is making things better but we still have a speed loss of about 50 %.
We’d love to use 2019R1 (and really hope for a point release to fix this bug) but unfortunately with this bug we’re stuck at 2018r3.

@Michael Diehr:
Good approach but only working with files?
I’m getting a RuntimeException:

First, there seems to be no problem on Mac.

Testing on Windows (W10, 1903) on both x86 and x64 builds, I am seeing the same issue.

My german is a little rusty, but with help from Der Google, I get this

The current process uses all the handles of the allowed high-precision window manager objects

This is interesting: in my production app, I’m actually not using Xojo.Core.Timer, but Classic Timers.

So it seems like we’ve uncovered a second, unrelated bug with Xojo.Core.Timers.

Here’s version 2 which only uses a single timer set to mode 2 (Multiple) rather than constantly resetting the timer to mode 1:

https://xochi.com/xojo/urlconnection/urlconnection2.zip

the bug with timers has been submitted as <https://xojo.com/issue/55529>

The bug seems to be that using Xojo.Core.Timer.CallLater or setting a classic Timer’s period within a thread is leaking resource handles. The workaround is to use a classic timer with mode=Multiple and create it outside the thread.

Good news, the <https://xojo.com/issue/55301> bug correction should happen in one of the next betas, Geoff confirmed that it would be tested. Concerning the bug <https://xojo.com/issue/55529>, there is finally no problem? or not since the R2?