File Writing Issue - Size Limitations in iOS?

Hey all,

My app needs to pull a database file from my desktop app in order to get it’s settings, etc. It does this once or whenever the user wants to update the database on the iPad. The file transfer and write is working fine for small files. Works quite slick and don’t have an issue at all. However, I just now tried to transfer the database one of my customers will use and I am getting failures. This database is much larger than anything I have tried before at 420 MB.

The file is base64 encoded for transfer and I am using the following code to save it to the iPad. I take the contents of the memory block from the TCP socket and convert it into a Base64 encoded Text variable. I then call the Decode and Write function shown below.

Dim Base64EncodedText As Text= TextEncoding.UTF8.ConvertDataToText(TheDBMemBlock,True)
Base64EncodedText.DecodeBase64AndWrite(appdbfile)

Thanks to @Jeremie_L for most of this code:

Public Sub DecodeBase64AndWrite(extends Base64EncodedText as Text, FileToWrite as Xojo.IO.FolderItem)
  // Thanks to Jeremy LeRoy for most of this.
  
  
  Declare Function initWithBase64EncodedString Lib "Foundation" selector "initWithBase64EncodedString:options:" _
  (obj_id As ptr, Str As CFStringRef, options As Integer) As ptr
  Declare Function alloc Lib "Foundation" selector "alloc" (clsRef As ptr) As ptr
  Declare Function writeToFile_ Lib FoundationLib selector "writeToFile:atomically:" (obj_id As ptr, path As CFStringRef, atomically As Boolean) As Boolean
  
  //Decode the Base64Encoded String as NSData (iOS Memoryblock)
  Dim mData As ptr = initWithBase64EncodedString(alloc(NSClassFromString("NSData")), Base64EncodedText, 1)
  
  //Write the file 
  //You will need to define path such as SpecialFolder.Documents.Child("some file").Path
  
  If writeToFile_(mData, FileToWrite.path, False) Then
    //all good
  else
    MsgBox "File Write Failed."
  End If
  
End Sub

So the failure is happening in the writeToFile_ method. There’s no error code or anything that I know of so how can I tell what is happening? Is there a file size or memory limitation imposed by Apple? This is my last step before final testing and releasing this to my customer. Like I say, with smaller sized databases, this all works fine. I’ve done it multiple times. But generally that has been with a much smaller file of about 23.5 MB.

So any help is appreciated.

Thanks,

Jon

I don’t know for sure but it seems like you are probably running out of memory with this approach. You probably need to do this in smaller chunks. Does the Low Memory event trigger when doing this?

Jason,

That’s a good question. Let me see what happens. I have not checked that event. I will now. How would I write a file like this size in smaller chunks?

Thanks!

OK. The Low Memory event does not trigger. I just tried this in the simulator. I added the low memory event and put a break statement in place. It never fired. The failure just happened.

Thinking about this some more.

It’s not working as it is written for large files. So I think, Jason, that you are correct that I need to break it into smaller chunks. Is there a way I can save each chunk as it comes over the TCP socket? Can I create the file and then just append to it each time?

Thanks in advance for any help…

So here’s what I am wondering…

Use the code provided by @Jeremie_L to decode the Base64 Encoded data:

Declare Function initWithBase64EncodedString Lib "Foundation" selector "initWithBase64EncodedString:options:" _
(obj_id As ptr, Str As CFStringRef, options As Integer) As ptr
Declare Function alloc Lib "Foundation" selector "alloc" (clsRef As ptr) As ptr

//Decode the Base64Encoded String as NSData (iOS Memoryblock)
Dim mData As ptr = initWithBase64EncodedString(alloc(NSClassFromString("NSData")), Base64EncodedText, 1)

Then put that ptr data into a memory bloc and then write it to a Binary Stream…

Dim mb as New Xojo.Core.MemoryBlock(mData)
MyStream.Write(mb)

I am thinking I can put this into my DataAvailable event in the socket and write it out with each chunk of data that comes in. Yes? No? Or am I risking mucking up my database this way?

Jon

Unless your checking you risk getting chunks in the wrong sequence because TCP is designed to behave that way.

Yeah, good point…

I wonder if I can read in the whole file and then break it into smaller chunks for saving a little bit at a time. Well, time to experiment I guess!

UGH! I hate it when I do something like this.

Total brain fart. I was running my desktop app from a compiled version that had a bug in the file transfer routine and didn’t properly encode the data. Running the desktop app from the Xojo Debugger causes everything to work just fine!

UGH!