I’m building my first full-fledged web application and I’m constantly bumping into smaller and bigger quirks, most of which I manage to resolve or work around. But today’s highlight is undoubtedly WebFile’s download, I’m at a loss with that one.
It always fails at around 40MB, no matter what.
Tried the web application in two different PCs, running Windows 10 and 11
On Chrome and Edge, the transfer resets and starts over when it reaches 40MB
On Firefox it just fails at that exact point
Now, for the test code:
Say I have a WebFile named file as a property in Session.
Then, there’s this code to fill it with data:
Session.file = new WebFile(false)
Session.file.Filename = "test.bin"
Session.file.MIMEType = "application/octet-stream"
Session.file.UseCompression = false // because if true, it takes forever to prepare
Session.file.ForceDownload = true
dim stream as new BinaryStream(Session.file.Data)
for i as Integer = 0 to 200
stream.Write(Crypto.GenerateRandomBytes(1000000))
System.DebugLog("data = " + Session.file.Data.Size.ToString)
next i
stream.Close
When it’s done, I have a button that just calls the Download method (or alternatively, GoToURL, makes no difference). Obviously, Session.file never goes out of scope.
What troubles me most, is that both the Forum and Cases are completely silent on this one, as far as I can see.
Am I the only one getting this behavior? Or am I the only one who’s trying to download files bigger than 40MB?
I just need your input/theories/suggestions/experience, before opening a bug report.
For the sake of focusing the discussion, I’m not interested in external workarounds (like saving the file someplace and then serving it via another web server), so please let’s keep it to what Xojo alone can and cannot do (and what it should be doing, but doesn’t).
I can’t reproduce your issue when running the app on macOS locally (2022r2 and 2023r3.1) or Linux on Lightsail (2022r2). Here is the demo project I built: download-test.xojo_xml_project
For what it’s worth, you don’t need another server to serve large files. You just need something running on the same server better equipped to serve large files than Xojo Web. Both Apache and nignx can do so, and it is recommended by everyone from staff members to expert users.
Xojo Web is not built to serve large files. That’s just the fact. Once you can see this example working completely, you’ll notice the “Session is having trouble communicating” message appear. This design will lock up the web app for all users.
Edit: Tweaked italics to highlight single server ability. Also wanted to edit in a plug for Lifeboat which can serve web apps and large static files from the same server (and even the same domain).
If you’re downloading by calling Session.GoToURL, try supplying the InNewWindow parameter value of True. The session (and thus the file) could be going out of scope when the browser navigates to the file. That doesn’t appear to be happening in my tests, but just something to try.
OK, it looks like something funky when running the application on Windows. I noticed that the download was painfully slow for a local operation (both client and server running on Windows 11, and connected with a macOS client). That was bad enough, but then the download on my macOS browser (Chrome) reached 80MB and restarted. Then it reached 191/192MB and restarted, then 68MB. Web on Windows with large files does indeed appear to be broken.
The server running on macOS works great in my testing, and appears to be working fine on Linux according to @Tim_Parnell . I’ll open a case for @Ricardo_Cruz .
Hey Tim,
Thanks for preparing a test project, tried it, failed in the exact same way my project fails. I’m solely on Windows.
On the “Session having trouble communicating”, it’s the first time I see it, haven’t come across it in any of my own tests. But I’ve never used a WebThread before.
On locking up the web app for all users, this is the reason that in my production implementation, I add
Thread.Current.YieldToNext
in the loop that feeds data in the file.data MemoryBlock. The other sessions aren’t completely happy with what’s happening, but at least they remain responsive. If I omit it and the loop lasts for a while, other sessions do lock up indeed.
On needing another server to serve large files, I don’t mean another physical server. When I say “server” I mean the piece of software that listens for incoming connections. I wouldn’t expect a Xojo web application to be serving large files as its primary function, but I would expect it to be able to serve an occasional larger-than-usual file, every now and then. Just enough so as not to complicate my architecture by depending on another (software) server (on the same machine). That’s not an unreasonable demand I believe.
Hey Anthony,
Thank god I’m not seeing ghosts
And thanks for creating the bug report.
My experience is that all downloads are slow via the Web Framework, it’s just more apparent in the case of big files.
I’ve never seen faster than 4MB/sec (locally or otherwise). There was even an early Web2 version that it was as low as 1,5MB/sec, but thankfully it got fixed (back to 4MB/sec that is)
I have managed to get speeds significantly faster than 4MB/sec, for example in this project: fswebapi
But at the cost of suffering what I call SYTH in that 2021 thread.
As I said back then: If someone took the time to review how things are done deeper inside the Web framework and the TCP socket, it might lead to an big improvement to how Xojo does networking. Might. I’m just saying…
But the consistently failing transfers in Windows is much more urgent right now: something’s clearly broken, not just underperforming.
First, thanks everyone for narrowing down the issue and @Anthony_G_Cyphers for opening the bug report.
There are two different issues going on:
Windows sockets seem to be working slower, compared to macOS & Linux.
The web framework is closing the socket too early, before the download finishes.
I’ve sent a fix for the second one, that will come in 2023r4. I’ve opened a separate Issue for the Windows socket transfer rate (Issue #74813). Even if it’s slower on Windows, at least the download will finish now.
@Georgios_Poulopoulos do you have access to beta releases? Could you please give it a try when the next build is available?
Hey Ricardo,
Thanks for your attention to the issue.
I currently don’t have access to the beta releases, but I’m going to renew my license later during the day, so I can try it out.
Meanwhile, I run some tests on a Linux Debian 12 VM and they weren’t exactly encouraging.
Using 2023R1.1 (win) I built a Linux executable and run it on the VM, on Firefox 115.4esr :
The transfer failed in the exact same way, so it’s not just a Windows issue
While it was transferring, data rate was higher than the Windows server build, but still not spectacular: 9MB/sec (localhost, just a single session).
Another thing I also verified happening on Windows too: The loop runs twice!
Meaning that I see the count “data = nnn” on the console and when it reaches the target size, it starts over once more!
There is a promotion at the moment in the store until Thursday, perfect timing!
The fix should cover that case also, it should work fine in the next available 2023r4 beta.
We’ve been doing performance improvements in 2023r4, but I don’t think you’ll see an increased transfer rate while serving a large file to a single session. What you can expect is lower average latencies and the capability to serve a noticeable larger amount of requests per second.
Please upvote and subscribe to Case #74813 to get updates about the slower transfer rate issue. We want to continue working on performance during 2024.
I’ve noticed the connection was being reset while working on Issue #74784. I can’t see that happening anymore in the internal build, but please give it a try to the next beta build when it comes available.
This is a very consequential improvement! Your attention to bug-fixing and performance issues is the reason I renewed my license at a time I was seriously considering alternatives.
Great job, keep them coming!
