File Access Speed with OSX Sandboxing

General
1

Hi

I have an app which I’m in the process of sandboxing (using App Wrapper). I notice that my sandboxed app now takes about 10 times longer to open/save/access files than the non-sandboxed app. Some more disk intensive activity in the app even seem to cause timeouts/app not responding.

The structure I have adopted for filing has the primary user files stored in SpecialFolder.Documents. On fileOpen these are copied to a temporary file in specialFolder.Temporary which is where the database is loaded from into the app. Database changes are therefore made into the temporary file. FileSave then copies the temporary file back to the primary folderItem location (having, of course, first deleted the old primary file). This structure therefore emulates the more traditional structure of only keeping changes to the file upon FileSave - rather than Apples newer versions.

The slow file access is particularly noticeable during regular database update activities (i.e. to/from the temporary file) rather than in the initial fileOpen, fileSave commands.

The questions are, therefore,

  1. is it normal that sandboxing slows down file access.
  2. Is the above the preferred way of saving databases
  3. Are there better ways of doing it that avoid sandboxing speed penalties?

Jim

2

No. But I’m not surprised that your method of copying files to other locations does.

Not to my knowledge

I believe the correct way is to use the SpecialFolder.ApplicationData.MyCompany.MyApp
location. This is then created as a sandboxed container within the Application Support folder.
Have a look at your ~/Library/Application Support/Containers location and where other sandboxed apps have stored their data.

3

Thank you Roger. That does indeed seem to have fixed it. For some reason access to specialFolder.temporaryData is much much slower than access to specialFolder.applicationData.

4

Do you have a Fusion drive in your Mac? The Apocations folder is stored on the SSD but I wager /tmp is not.

5

Unfortunately I spoke to soon. It appears another mod I made to the programme is what made it appear to run faster. The mod reduced the amount of file reads, but I now find that in areas where file writes are required, it is slow as hell.

.temporary and .applicationData seem to run at the same slow speed when in Sandbox - but fly when running outside the sandbox (using the same destinations).

Yes I do have SSD, but it seems that both drives are slow.

Jim

6

If you want to provide an auto save mechanism, there’s an app, I mean folder for that! Look at ~/Library/Autosave Information. I wouldn’t autosave data to the temp folder, the reason being is that the tmp folder (used) to get flushed on restart, so if your auto save data is there and the user restarts or the machine crashes… It’s gone.

Depending on what kind of application that you’re working on, it might be okay to simply write to the database when changes are made.

7

This problem continues to bug me but it looks like its not so much of a Sandboxing issue as a CodeSigning issue. If I turn off codeSigning in AppWrapper the problem goes away - file access is at full speed again.

So I thought I’d do a smaller scale test programme, accessing a simple sqlite file in 4 different SpecialFolder positions - deskTop, documents, temporary and applicationData, and looping through read/writes to test the access speed.

When I run in Sandboxed only, it accesses all 4 file areas without problem, and at more or less the same access speed. When I turn on CodeSigning (with a valid certificate refreshed in xCode), access to all destinations is denied, except for Temporary - which always seems to work fine.

The part of the code that fails is marked below “//<<first error message here.” The error is “unable to open database file”. i.e. it is able to create the database file, able to connect to the database file, but as soon as I try to write to it, it fails.

It works fine in debug, runtime and sandboxed versions, just fails in CodeSigned version.

This is a bit different to what I was finding in my app, but I need to get over this hurdle first.

Anyone have any ideas?

Jim

[code] dim DB as SQLiteDatabase
db = new SQLiteDatabase
dim filename as string
filename = “test.sqlite”
dim f as FolderItem
//create a new file
select case type
case location.desktop
f = SpecialFolder.Desktop

case location.documents
f = SpecialFolder.Documents.child(“actio Temp”)

case location.temp
f = SpecialFolder.Temporary

case location.applicationdata
f = SpecialFolder.ApplicationData

end select

//need to save it
dim s as new SaveAsDialog
s.InitialDirectory = f
s.SuggestedFileName = fileName
s.PromptText = “confirm save”
dim savedFile as folderItem = s.ShowModal

if savedFile <> nil then
db.DatabaseFile = savedFile

//add to recent files
select case type
case location.documents, location.desktop
  addTo_NSRecentFiles(savedFile)  //Sam Rowlands code from appWrapper Help, adds to NSDocumentController recent files.
else
  //no need to add
end

else
msgBox(get_Name(type) + “: file not working” )
return nil
end if

if db.CreateDatabaseFile then
//create a table entry

if db.Connect then 
  dim sql as string = "Create table Test(GUID Integer Primary key, tVal integer)"
  db.SQLExecute sql
  
  if db.Error then //<<first error message here.
    MsgBox get_name(type) + ": creation eror: " + db.ErrorMessage
  end if
  
  return DB
else
  MsgBox get_name(type) + ": cannot connect: " + db.ErrorMessage
end if

end if

return nil[/code]

8

Just to confirm one thing, Sandboxing is part of the code signing process. So your application will only be sandboxed when it is code signed.

#1 Try using a folder with the Application Support folder.

Dim appDataFolder as folderitem = specialFolder.applicationData.child( "com.mycompany.myapp" ) if appDataFolder.exists = false the appDataFolder.createAsFolder

#2 Just after you created the database, try disabling the file journal.

db.SQLExecute( “PRAGMA journal_mode = MEMORY;” )
10

To finish this off, I finally found the causes of my woes.

  1. The slowness when Sandboxing my app was caused by the RecentDocumentURLs method, which was being called by EnableMenuItems events. I used the Declare functions prepared by Sam in his help files for AppWrapper, viz:

[code]

Dim rvalue(-1) as string
Dim n,l as integer

// - While this method only includes code for Cocoa, it’s been designed so that it can be expanded to
// - support additioanl targets.

#if TargetCocoa then
// - This function will get a list of the recent items from the NSDocumentController, as an NSArray.
declare function getURLS lib klibrary selector “recentDocumentURLs” ( docController as Ptr ) as Ptr
Dim recentDocURLs as Ptr = getURLS( sharedDocumentController )

// - How many items are in the NSArray?
declare function getCount lib klibrary selector "count" ( ref as Ptr ) as integer
n = getCount( recentDocURLs ) -1

// - The following two declares get the item from the NSArray, then get the URLPath from the NSURL object.
declare function getObjectAtIndex lib klibrary selector "objectAtIndex:" ( ref as ptr, index as integer ) as Ptr
declare function absoluteString lib klibrary selector "absoluteString" ( ref as Ptr ) as CfStringRef

// - Now we loop through and extract the URLPath (which we can use in Xojo).
for l=0 to n
  rvalue.append absoluteString( getObjectAtIndex ( recentDocuRLs, l ) )
next

#endif

Return rvalue[/code]

However, for some reason, when Sandboxed, this routine takes about 90msecs to run, compared to just a couple of msecs outside sandboxing. So the solution was to just run this routine on Filing events (close, save, saveAs, open, openRecent), storing the result locally to be called by enableMenuItems.

  1. Saving to and from an SQLite file in SpecialFolder.Temporary and SpecialFolder.ApplicationData runs just fine. But trying to access the file within SpecialFolder.Documents or SpecialFolder.Desktop, won’t run in Sandboxing because SQLite creates its own temporary files which don’t have the necessary sandboxing permissions. This has been reported elsewhere in the forum but the challenge in detecting it is made difficult by the db.connect function which returns true, whilst subsequent db.sqlExecutes etc will fail to connect to the database (because they are trying to connect to a different file).

I didn’t try Sam’s suggesting of disabling the Journal file - so don’t know if that works.

  1. My original solution of copying the database file from Documents to Temporary or ApplicationData and doing all database operations there runs fine, though I’ll move to using the ApplicationData folder as it has more permanence for recovery of lost files.

  2. My confusion over Sandboxing vs. CodeSigning was generated by the fact that AppWrapper has 2 independent checkboxes for these functions. Perhaps the sandboxing one should only be enabled if codeSigned is true.

Thanks for your help.

Jim

11

Oh my… How embarrassing… I will look into this as soon as possible.