WebFileUploader critical bug in 2021R1.1

Mike_D · April 26, 2021, 3:47pm

Danger Danger!

The bug where WebFileUploader was adding garbage to the end of an uploaded file was fixed during the 2021R1.1 betas, but appears to have shown up again in the 2021R1.1 release version.

This is a critcal bug and a 2021R1.2 will have to be released to remedy this, as there is no workaround.

Regression testing of prior builds:

Build # : Behavior:

2021 R1.1 (release) : uploaded file is truncated
2021 R1.1 (52913) : uploaded file is truncated
2021 R1.1 (52906) : uploaded file is truncated
2021 R1.1 (52865) : uploaded file is truncated
2021 R1.1 (52841) : uploaded file has trailing garbage

<https://xojo.com/issue/64352> should be reopened.

MarkusR · April 26, 2021, 3:52pm

oha …

Mike_D · April 26, 2021, 3:57pm

I’m confused by how this happened, as I was testing the betas and thought it was fixed. Weird. In the 2021 R1.1 release version (and prior betas) it looks like the file is getting truncated at 1MB approximately.

In any case, if you are using WebFileUploader and need this to work today, you can go back to 2021 R1 and use the hack here: WebUploadedFile Trouble - trailing line added - XML trouble - #4 by Mike_D

Mike_D · April 26, 2021, 4:02pm

The problem seems to affect larger files, but not smaller ones:

I think I understand it now - there are basically 2 different bugs:

file has extra garbage at the end
file is truncated too short

It looks like 2021R1.1 fixed bug #1 (with all size files) but introduced bug #2 (but only with large files).

Since we were all testing with short files looking for bug #1, nobody noticed the introduction of bug #2. Drat!

Rick_Araujo · April 26, 2021, 4:06pm

Don’t trust just in the file size, get a hash of the file (like sha256 or md5) on origin and check against what you find at destination. I’ve found systems fails in the past with correct sizes but buffer garbage at end (or middle).

Rick_Araujo · April 26, 2021, 4:15pm

I requested to reopen it.

MarkusR · April 26, 2021, 4:15pm

somehow this WebFileUploader need a unit test with different browsers and files
with use of checksum validation.
the files() must also processed at UploadFinished because outside they will gone/deleted.

Mike_D · April 26, 2021, 4:22pm

I submitted a new case with improved logging, and small and large sample PDF files to show the bug: <https://xojo.com/issue/64552>

Christian_Schmitz · April 26, 2021, 5:59pm

Can you guys look in temp folder for temp files?

Xojo writes the whole request in a temp folder, so you may be able to copy it.
Maybe Xojo has a parse error for the length there?

Mike_D · April 26, 2021, 6:18pm

I checked and see only some zero byte files like this:

ls -l /private/var/folders/[...]/T
-rw-r--r--      1 user  staff        0 Apr 26 11:14 (null)-temp1676375564
-rw-r--r--      1 user  staff        0 Apr 26 11:13 (null)-temp6148852

Christian_Schmitz · April 26, 2021, 6:23pm

Well, I can confirm the bug here with the standard example from Xojo.
I just changed it to log the size it received:

Christian_Schmitz · April 26, 2021, 6:45pm

I made a new case 64558 and I will collect my details here.

Christian_Schmitz · April 26, 2021, 9:05pm

I hope Xojo Inc. can fix that quickly and maybe push us a 2021r1.2 or 2021r2 beta as upload seems to be broken.

I don’t see much of a workaround except getting the temp file and splitting the mime parts ourselves to get the file data.

Jeannot_Muller · April 27, 2021, 11:21am

Yep, urgent as many web apps (at least in my case) depend on (reliable) uploads.

Mike_D · May 2, 2021, 3:13pm

Bump - it’s nearly a week later and <https://xojo.com/issue/64552> still shows as “needs review” - @Greg_O_Lone this is a pretty bad regression which corrupts data - it would be good to hear about Xojo’s plans to deal with it, hopefully a quick R1.2 release?

Ivan_Tellez · May 2, 2021, 4:48pm

The first rule of being a Xojo user: NEVER EVER use a new release to compile a end user app, you have to do all the QA in your software that Xojo does not do, even wait for others to report some problems. Otherwise you will end with broken software on your side without even make changes in your code.

Second rule, never hold your breath for a patch

Jeannot_Muller · May 2, 2021, 5:23pm

Where did I write anything about releasing an end-user app? . I said, and I am happy to repeat that most of my apps need upload functionality, and I dare to conclude that most people have solutions in mind where they will need a trusted upload functionality as well to solve a customer’s problem.

For me there is still https://graffitisuite.com/ @Anthony_G_Cyphers’s uploader is working, so no worries.

But this plan B doesn’t change my opinion that a viable web solution needs rock-solid upload functionality. But I’m aware as well that it isn’t an easy task to realize. Many programming solutions have and had challenges with this.

Greg_O_Lone · May 2, 2021, 5:34pm

[SQL]
SELECT holiday FROM xojo WHERE NOW() BETWEEN ‘2021-04-28 17:00:00 EDT’ AND ‘2021-05-04 08:59:59 EDT’;
[/SQL]

Christian_Schmitz · May 2, 2021, 6:25pm

Happy Easter to my Orthodox friends!

Jeannot_Muller · May 6, 2021, 6:25am

I noticed yesterday some interesting behaviour. Code working perfectly fine on macOS and Linux (for small files around 250 KB max and compiled with 2021 1.1). fails on a Window Server in all type of browsers. Compiled the code with release 2021 1 instead of 2021 1.1 and it worked perfectly fine again for those small files (didn’t test for large files).

Side note: It seems the release 2021 R1 is no longer available for download? At least I didn’t found it. I think I remember that Xojo is not keeping hold of older releases when a point release is published, but this might cause some trouble for users in such circumstances. But perhaps I have just overlooked it.