webfile we download

I just noticed to my supprise that when a webfile is being downloaded the complete WE server stops until its complete. This means if a user downloads a 100mb file the whole server will just stop for ages.

I tried to move the code into a thread but with XOJO you cannot access the session object. How can I download a file without locking up the web engine?
Dave

@dave duke
Don’t use WebFile for things that big unless you absolutely must. For instance, if you’re using a CGI web app, have the web server (i.e. Apache) deliver the file.

There are two problems with what you’re trying to do. The first is that WebFile currently lives entirely in RAM. So that 100MB file consumes 100MB of memory, and when a user requests it, the framework copies it for transfer to the socket (so it takes at least 200MB for a little while). The second issue (the one you’re running into) is that the framework currently writes the entire “file” to the socket at once, which is likely where the hang is occurring.

It’s on our list of things to fix.

[quote=26359:@Greg Olone]There are two problems with what you’re trying to do. The first is that WebFile currently lives entirely in RAM. So that 100MB file consumes 100MB of memory, and when a user requests it, the framework copies it for transfer to the socket (so it takes at least 200MB for a little while). The second issue (the one you’re running into) is that the framework currently writes the entire “file” to the socket at once, which is likely where the hang is occurring.

It’s on our list of things to fix.[/quote]

indeed, it is a very important issue.

Maybe the JQueryFileUpload control (for Xojo) solves the problem?

http://webcustomcontrols.com/?page_id=70

These are controls of high quality, easy to use.

I don’t know. You saw that WebFile bugs have been fixed in the last beta?

https://forum.xojo.com/3910-webfile-change

https://forum.xojo.com/3904-webuploadedfile-changes

Have a look at <https://xojo.com/issue/19124> & <https://xojo.com/issue/28528>. These are from the R3.0 release and allow you to spool uploads & downloads.

We did some work on WebFile so that it’s possible to leave the contents on disk. Calling WebFile.Open has an optional parameter which lets you specify if the file is to be loaded into memory or not.

Anyway, the advantage is that now WebFiles can be spooled directly from disk, in 64K chunks, reducing the memory required significantly. Previously, the memory used would be 3x the size, now even an in-memory WebFile should only use 2x at most.

If they had “Xojo Awards”, this would win “Best. Fix. Evah.”

Is this still true?

If so that is pretty bad.

What is the workaround to prevent locking up the WE server for other users if one user is downloading a file?

I recently purchased WE but it seems to have many problems which aren’t solved yet and I’m not sure if I should get a refund?

As of r3, WebFiles are spooled in 64K chunks.

[quote]As of r3, WebFiles are spooled in 64K chunks.
[/quote]

Hi Greg and thank you for replying.

I understand that will prevent the huge ram use which is great but I don’t understand what that means to the problem of locking out all other users if one user is downloading a file?

Will other users still be basically locked out or denied server use while a large file is being downloaded?

Can you please be exact.

Perhaps you have a feedback report on the issue of “locking out other users” or “denying server use”? This is contrary to my own experience with Web Edition, even with large files. Is that exact enough for you?

Hello and thank you for your reply Brad.

I personally haven’t spent the time trying this out (busy desktop developing) but I was going off other’s posts with this experience because I’m trying to learn about WE’s limitations and abilities in my extra time.

From the OP:

If you’re saying there is no issue where large files or files of any kind being transferred temporarily ‘lockout’, deny/hang server use for other users then why would Greg explain it?

From Greg:

[quote]
The second issue (the one you’re running into) is that the framework currently writes the entire “file” to the socket at once, which is likely where the hang is occurring.

It’s on our list of things to fix. [/quote]

I also thought I’ve previously read about this issue many months ago here on the forum but I could be wrong.

If it works correctly and all other users are unaware that a user is file transferring that would be great but from what the OP and Greg wrote it doesn’t seem that way.

Ok, so you’re stating it as some kind of fact and don’t have any real experience with it, either in the past when it might have been a problem, or now, when WebFile is spooled out in chunks.

Here is what I can tell you as fact, based on having a product that real people use to share large files, made with Web Edition. With RealStudio 2012r2.1, deployed on a Linux VPS with enough memory resources and plenty of outbound network bandwidth, writing a large file (100MB) to the socket does not hang up the WE app. The 3x memory use issue and a related compression issue are far more problematic for large file downloads. These problems are both fixed as of Xojo 2013r3.

If, with RealStudio 2012r2.1, you need to work around the 3x issue or your actual experience with WebFile download hangups indicates that you need to take some action, it is not a difficult project to hook up a “download” directory with Apache on the same server, where you write files, serve them up with Apache, and delete after a reasonable period of time. With any Web Edition app, you might also consider having Apache serve up static images and other large files if you want to squeeze the most performance out of a WE app. If it’s always busy serving static images, it’s not busy handling real user interactions.

Finally, please contact support and get your screen name changed. We’re all friends here.

HI Brad, I appreciate your time and effort, so thank you.

It’s true that I don’t have any personal experience with this issue on web edition since I’m new to it and haven’t played around with it very much yet. But let’s not ignore this issue/complaint was from the OP -another more experienced web edition customer than I am who is experiencing this problem. Let’s also not ignore the actual developer Greg verified the problem and even said it’s

So it’s not like I’m just making up this concern without any foundation. When I read one customer has an issue of something not working correctly and then I read the actual developer confirms it and then verifies it’s on the list of things to be fixed I pretty much go with that until I have time to test it myself. We should also note that being ‘on our list of things to fix.’ doesn’t mean it will get fixed any time soon which is cause for further concern. There’s no time table for customers to go by so it’s very hard to know if a solution is near or not.

If you’re saying it works great in previous RS versions that’s good news but maybe it’s not working correctly in xojo versions? I’m not sure I’d be entitled to those previous RS web edition licenses since I bought when it was named Xojo? Even if I were I’m not sure I’d want to go backwards in versions and suffer other issues but it is good to know prior versions are working so thank you.

I’m now wondering if this problem also affects client uploads because I do have a project in mind where clients will need to perform various uploads (and downloads)?

I could write it in php or another web technology but that kind of defeats the entire purpose of me purchasing web edition. If I have to have a half web edition half php/html5 site then I’d just rather go will full html5 / php. When I have more time I’ll do some testing.

That’s great to know and thanks. However, I didn’t sign up with the name you see. ‘User U’ nor ‘User Unknown’ or whatever it currently says.

‘Someone’ at xojo inc. changed my name to ‘User U’ or whatever it is now. I suspect it’s the Xojo inc. employee that often gets into heated disagreements here on the forum with customers. He’s pretty much outwardly said he doesn’t respect customer’s privacy so I figure he’s the one who changed my name into what you now see in an effort to dissuade other’s from helping me (speculation).

Who at Xojo inc. changed my name is speculation but the fact that someone or something at Xojo inc. changed my name to what you see is fact.

I’ve been a paying customer for many years with this company and I prefer to maintain my privacy. There’s nothing wrong with that and it would be proper if that was respected :slight_smile:

I think intelligent quality people won’t hold the ‘User U’ name against me which is why I haven’t created a new name :slight_smile:

You Brad, Greg and others have already proved that to be correct so thank you.

In the event that some people don’t want to help others because they do not respect their privacy I think that’s very unfortunate.

[quote=42895:@User Unknown]So it’s not like I’m just making up this concern without any foundation. When I read one customer has an issue of something not working correctly and then I read the actual developer confirms it and then verifies it’s on the list of things to be fixed I pretty much go with that until I have time to test it myself. We should also note that being ‘on our list of things to fix.’ doesn’t mean it will get fixed any time soon which is cause for further concern. There’s no time table for customers to go by so it’s very hard to know if a solution is near or not.
[/quote]

Ok. Well, you just basically sound unhappy. That’s too bad. My experience has been that when I tracked down some of these problems being discussed here and reported them completely enough in Feedback, Greg got on them and fixed them in a very timely manner. If you run into an issue and can spend time articulating it here, you might get some very useful advice. The OP really doesn’t give us much specific to work with.