Sandboxed Mac apps die when your select more than 3,700 files

Hey everyone.

I’ve got a Mac OS X application where I copy files to the app’s folder in Application Support and do some conversion, then copy the files back to a destination. Everything works fine until I sandbox the app. Then, when the number of files is in the thousands… it will stop copying around 2700 files and ignore the rest. This happens in all flavors of XOJO and later version of Real. It happens irregardless of wether I use Applescript and shell commands to do the copying, or XOJO code. Works great during tests and builds, but breaks after sandboxing.

Anyone else seen this? Any work-arounds?

Thanks!

What’s the file copy code look like?

for i as integer = 0 to dlg.Count-1 f = dlg.Item(i) f.CopyFileTo (dir) next

[quote=48542:@Niles Mitchell] for i as integer = 0 to dlg.Count-1 f = dlg.Item(i) f.CopyFileTo (dir) next [/quote]

Is dlg a FolderItem?

dlg is a new open dialog

When copying stops, what is f's LastErrorCode?

I don’t know how to check that.

Look there:

http://documentation.xojo.com/index.php/FolderItem.LastErrorCode

it will do something like:

[code]Dim i As Integer
Dim TheCnt As Integer

TheCnt = dlg.Count-1

For i = 0 To TheCnt
f = dlg.Item(i)
If f.LastErrorCode > 0 Then
MsgBox("[i]: "Str(f.LastErrorCode))
End If

f.CopyFileTo (dir)
If f.LastErrorCode > 0 Then
MsgBox("[DIR]: " + Str(f.LastErrorCode))
End If
Next[/code]

I put code in to present a message box for errors, as suggested. I’m not getting an error codes back. It just sails on like nothing went wrong, except that it stops copying around a little over 3700. Not sure why it’s doing 3700 instead of 2700 now, but that’s what it’s doing.

Again, this works perfectly when not sandboxed…and also get the same problem if I use shell scripts to copy the files, so it’s not just XOJO code. This is something with Sandboxing.

Somebody else has to has come across this before…

Is it AppNapp? You can turn it off in GetInfo.

No, it only happens when it’s Sandboxed.

OK, so I’ve re-written and optimized my application. I am no longer copying the files to a temp folder before doing the conversion…I’m doing it right on the fly. Guess what? This thing still craps out around 3,700 files when sandboxed. This is not just a copying bug. The bug seems to be when the dlg.count is over roughly 3,700 while the app is sandboxed.

Any ideas? Can someone confirm this? Sandbox your app then select let’s say 4,000 files to do something. Curious is yours craps out too…

You’re letting the user select 3700 files in the open dialog?

Having you tried a drag and drop mechanism instead?

It seems to me, that this might be an unexpected limitation of the Sandbox, I’m guessing that every time you ‘gain’ access to a file, that ‘access’ is retained.

It’s a pretty large amount of files, maybe something that Apple wasn’t expecting… One would like to hope not!

Can you add some code to check the following on each file.
.Exists
.isReadable
.isWritable

Maybe that will help, I don’t have a single folder with that many items in it, or any you selecting photos from iPhoto or something?

It’s an image converter program, so it’s quite feasible that an end user could select thousands of files.

Not sure about your request for adding that code. These files do exists. It’s not the copying, as I mentioned earlier, that is causing the problem. It’s just the dog.count that’s making it puke. Remember, these same files work non-sandboxed.

Sounds like a bug Shiny Frog has found…
http://help.shinyfrog.net/discussions/pixa/1559-is-there-limit-to-number-of-live-folders

[quote]Christopher: However, after about 2300 graphics files in about 48 Live Folders, Pixa has stopped being able to add Live Folders.

Matteo: unfortunately there is a big bug in the Apple Sandbox that prevent apps to load files after a certain number as already be opened (that’s why quit/restart fix it).
We’ve already reported that to Apple, but we’re still waiting for a fix, hopefully it will come in 10.8.4.[/quote]
And here’s a demo of the bug they made. No notes in there, but maybe you can get something out of it.
https://github.com/shinyfrog/sandboxbreaker

[quote=50208:@Niles Mitchell]It’s an image converter program, so it’s quite feasible that an end user could select thousands of files.

Not sure about your request for adding that code. These files do exists. It’s not the copying, as I mentioned earlier, that is causing the problem. It’s just the dog.count that’s making it puke. Remember, these same files work non-sandboxed.[/quote]

Did you try to copy the files in two rounds of say 2000 ?

[quote=50216:@Will Shank]Sounds like a bug Shiny Frog has found…
http://help.shinyfrog.net/discussions/pixa/1559-is-there-limit-to-number-of-live-folders

And here’s a demo of the bug they made. No notes in there, but maybe you can get something out of it.
https://github.com/shinyfrog/sandboxbreaker[/quote]
Yikes, if that’s the case then chances are there is a limitation on how many files the app can hold onto references per session… In which case some creative thinking (and multiple sessions) would be a workaround…

[quote=50208:@Niles Mitchell]It’s an image converter program, so it’s quite feasible that an end user could select thousands of files.

Not sure about your request for adding that code. These files do exists. It’s not the copying, as I mentioned earlier, that is causing the problem. It’s just the dog.count that’s making it puke. Remember, these same files work non-sandboxed.[/quote]
My suggestion is to be able to probe the file with common methods to see if there is a way you can figure out when you’ve reached the point of not being able to process (especially if functions fail without an error message).

Perhaps a helper app? If done right, it would share the sandbox of your parent app. You could run one helper per thousand files, for example. I’m just spitballing, I’m not sure if it would work, or if the bug/limit would apply to the entire shared container.

Thanks guys! I’ve thought about trying to split it up into groups of one or two thousand, but not sure quite how to do that. What I have tested that works is a simple message box that pops up when the user has selected more than 3,000 files and politely requests they try a smaller batch.