FolderItem.exists or .length incorrect immediately after BinaryStream write

Targets macOS
1

I have an app which makes 1000s of files, using the following pseudo-code:

fCacheFolder=SpecialFolder.UserLibrary.Child("com.myapp").child("Cache")
for i as integer = 1 to 10000
  dim data as string = ...  
  dim f as folderItem = fCacheFolder.child("file " + str(n) + ".data")
  dim bs as BinaryStream = BinaryStream.open(f, true)
  bs.write data
  bs.close
  bs = nil

   if f.exists = false or f.length = 0 then
     // ERROR
   end if
next

On macOS, this fails about once per 1000 files. When that happens, looking in the debugger, I see that indeed, f.exists = false (and/or f.length = 0) even though I can look in the finder and the file is on disk and non-zero length.

A work-around which solves the problem:

   if f.exists = false or f.length = 0 then
        // this may fail even if the file exists.  try again
       f = nil
       App.sleepCurrentThread(10)
       f =  fCacheFolder.child("file " + str(n) + ".data")
       if f.exists = false or f.length = 0 then
              // ERROR - for real this time
       end if
   end if

I’m using Xojo 2014 R2.1. I feel like these errors are more likely to happen in macOS 10.12 than 10.10 or 10.11.

My question: is anyone else having this problem? The kludge seems like it works OK, but I’m not sure why this is necessary.

2

Before bs.close, insert:
bs.Flush

That said, if you can, try your application on a local AppleStore Macintosh. Why ? Because their hard disk OS cannot be virused, the Finder refresh rate cannot be changed (try: cmd-n, then type as fast as possible a name for the newly created folder: if the first(s) characters are not there, you are in trouble…)

If the number of created file is really 1000 to get troubles, write a simple AppleScript who will create and populates 1000 text files.

Did you checked (using Disk Repair) your hard disk (the one you wrote in ?).

I suppose that you leave Xojo, close all the applications, set WiFi to OFF, shut down, wait a couple of minutes and boot. Then fire the application and watch what happens.

3

Thanks Emile - is calling bs.Flush a known fix or are you just guessing?

4

Both.

http://documentation.xojo.com/index.php/Writeable.Flush

I used it far before the ARPA was know by the general public as Internet.

I recently used once or twice with Xojo and my project was working fine since that.

At last, it takes only seconds to verify it this works in your specific case.

5

After fixing some of the problems, I’m also seeing this fail with a simple FolderItem.Create call, e.g.

  dim f as folderItem = g.child("xxx")
  if not f.exists then
        f.createAsFolder
  end if
  if not f.exists then
      // oops!
  end if

so I’m wondering if this is less a problem with BinaryStream and more a problem with FolderItem itself?

6

Writing is not instant on a Mac. you should allow some delay.

7

Whats the path to g ?
Is is protected in a way you require elevated permissions to create ?
What is last error ?
What happens if you use a newer version as this could be a bug that has been fixed or something that has been updated in newer versions) ?

8

First of all, using g as folderitem reference name is suspect :frowning:

9

Ah there you go
It should always be F not G :stuck_out_tongue:

10

Yes, but f is already used. Maybe f1 ?

11

that might be too fast for some :slight_smile:

12

SpecialFolder.UserLibrary.Child(“com.myapp”).child(“Cache”).child(“abcdefg1234”) // the actual name is a 16 digit unique hex string.

Nope - as mentioned this works fine 99.99% of the time but fails about 1 in 1000.

No errors are seen

I have not tried and am stuck with 2014R2.1 for reasons.

One additional point I should add: this is all being done inside a thread.

Also, I should emphasize that this was never a problem in os x 10.9, 10.10, or 10.11. We really only started seeing this particular issue in 10.12. When it happens, nil’ing the folderItem, waiting a few milliseconds, then re-creating it always seems to “refresh” the folderItem’s values so they are correct. It’s not that the file write failed, it’s just that the status of the FolderItem is incorrect.

If I had to guess, I’d suspect that 10.12 has created some new problem with Xojo folderItems, where (on occasion) they are not in sync with the filesystem. It’s very rare (about 1 in 1000) which could suggest a timing issue, but deterministic (e.g. it always happens on the exact same loop iteration).

13

OR that certain functions in the Xojo framework (ie/ Carbon based file API’s) have been updated as newer OS versions have come along and that its possible that older deprecated API’s in use by older Xojo versions are finally failing or have bugs that Apple wont ever fix

And that sticking with the old versions will never permit you to resolve those issues

Its why I suggested trying a newer version to rule that out but …

14

[quote=337955:@Norman Palardy]OR that certain functions in the Xojo framework (ie/ Carbon based file API’s) have been updated as newer OS versions have come along and that its possible that older deprecated API’s in use by older Xojo versions are finally failing or have bugs that Apple wont ever fix

And that sticking with the old versions will never permit you to resolve those issues

Its why I suggested trying a newer version to rule that out but …[/quote]

A very reasonable suggestion! I’m going to try to come up with a small demo app which shows the problem, assuming I can do that in 2014R2.1 then I’ll try it again in 2017. Since my current work-around is stable, this is not critical for me.

But I wanted to alert others in case this is not something that’s fixed in Xojo 2017 and is happening due to bugs in 10.12 etc.

15

and lord knows what 10.13 and APFS are going to do to everyone

16

I have a friend who recently upgrade its PC (from Windows XP to WIndows 7)…

17

Moved from “completely obsolete” to “mostly obsolete”
Not much of an “upgrade”

18

Also, although I’m using 2014R2., this is a fully Cocoa app. Would Xojo still be using Carbon functions in that case?

19

What I can say as an oldster (and was already by young Norman) is… if you do not upgrade your application (REALbasic, Real Studio or Xojo), and you update your OS, at a point in time your application will start to have troubles until it will stop to work.

20

This is somewhat true on macOS but is categorically not true on Win32 - I’ve got apps built in RB2008 which work fine on Windows 10, for example. Microsoft and Apple have a very different approach to backwards compatibility.

In any case, the question I asked is not “is it a good idea to upgrade Xojo” but rather “Is anyone else seeing these specific problems.”