WebFileUploader Fix

Like many others, I’ve been having trouble with unreliable WebFileUploader. Although small uploads usually work, larger uploads routinely fail, and the issue seems to be related to the network speed and/or latency, e.g. the exact same file upload will succeed on fast WiFi but fail over a slow cellular connection.

Feedback Case: feedback://showreport?report_id=61753 (this is visible to Testers only)

The failure has a very similar pattern:

  • UploadProgressed hits 100%
  • UploadFinished never fires
  • In the /tmp folder, the uploaded file is present, but is about 90% complete.

I’m pretty sure this is a bug inside Xojo with socket or multipart-mime handling.

In any case, I realized that the file is mostly there, it’s just the end bytes that are missing.

So here’s the hack:

  • Override the XojoUploaderEngine by using Javascript inside of Session.Open. The altered javascipt simply adds a second copy of the upload file to the upload queue.
  • On the server side, whenever UploadProgress hits 100%, start a timer
  • When the timer fires, if UploadFinished hasn’t fired, then we go hunting inside the /var/folders temporary items to find the partial upload. Repeat as necessary
  • Because we included the same file twice in the upload queue - only the second copy of the file upload is truncated. The first copy is fine. With some multipart-mime hacking, we extract that first copy and save it to disk.

This is a super ugly hack, but I’ve been testing it in production for 48 hours, and it has reduced my # of failed uploads to zero.

V1 Demo Project available here: https://xochi.com/xojo/WebFileUploader/

Related threads:

Update: I’ve been running this fix for about 48 hours, and so far it’s performing perfectly. Of approximately 200 uploads, about 65% of them failed initially. However with the fix in place, in every case the upload failed, the file was able to be recovered successfully. End result: 100% success rate on uploads.

The uploads are Word and PDF files ranging from a few KB up to about 5MB in size, with a mix of users using macs, windows, android and iOS devices on their home networks.