When is file copied

I have a situation that I copy very large files from a NAS to my main SSD drive.
I created a tool that triggers an event when a file has been added to that certain folder.

Now, is there a good solution to know when a file has been fully copied? I could add a timer and check if the file is still changing in size but that’s a bit too much. I guess there must be a must easier solution.

Copy to a temp folder, then move it?

Although that would be a good solution, it is not possible in this case.

What about to a temp file within the target folder? Or a subfolder within the target folder?

Or try to open the file for write. If you get an error, it’s still being copied. (This will require testing to confirm.)

How about you copy it yourself with a binary stream? You’d be able to provide a precise progress and know exactly when it is finished, you could also control the speed by deciding how much or how little data is written in a single chunk.

Unless you are in control of the process that actually copies the file, then no, there is no simple way of knowing that the copy is complete. If you could predict where the file is coming from, you might be able to compare the original file size to the one on your SSD and tell if the copy is complete. Otherwise, you have to use a timer to tell if the file size is still changing.

Use an AppleScript in the target Folder that send an event “NewFile” (is available) to your application ?

OK macOS only, but Script attached to a Folder is possible (but I never wrote one).

Just an idea to test.

Or, in a Timer, poll the Target Folder and send an Event “NewFile” (is available) to the application…

Yes, I do not have any control over this.
As far as I know the only way is to check if the file size is growing. If not, for example after 5 seconds, it is copied. But this not 100% for sure. Especially if the file is copied from a NAS and the netwerk is slow.

On M1, when I have the bad idea to copy a file onto a MemoryStick, there is latency before the copy start and the copy process is… slow, very slow.

If you only need a macOS solution, you can use FSEvents to monitor a folder and be notified when a file is added (or removed, etc.). I use this to create a “watch” folder that notifies my app when a PDF is added so it can be processed. There is FSEventsMBS class as well as a similar class in MacOSLib. The MBS solution is detailed here

https://www.monkeybreadsoftware.net/class-fseventsmbs.shtml

On Windows I attempt to rename the file. If it succeeds then the file is complete. You can rename it twice assuming the file has the correct the name when copied.

With a timer you should count the number of attempts and signal an error after some reasonable period of time. Without this, if the other process gets stuck, your process also gets stuck.

This may not apply to your situation, but another trick that I’ve used with ftp. I need to process files that have been transferred in, so I need to know when the file is complete. Many ftp servers can run a script after a transfer completes. I have the script create another file to indicate that the real file is ready.

I know but that will trigger even if the file is not yet fully copied. So you still need to know when it is fully copied/available.

Did you try opening with Write permission to see if that will give you an error?

1 Like

It only fires when the file is completely written to disk. As I said, I use this extensively for processing PDFs, which I do as soon as the notification arrives.

That is not the case (100% for sure). I know that API inside/out. :slight_smile: It also triggers when the file is not copied fully.

For small files, this is not important but for files that can take seconds or minutes, you still need to check if it is fully copied.

That’s a good question and something I will try!!

I would just have a timer wait till the last modification date is 10 second in past and file size hasn’t changed for 10 seconds or so.

That’s a solution too. I already tried successfully checking if the file has write permissions and that’s working perfect!

Thanks Kim and Christian ! Both are very good solutions!

@ Jonathan:
If you do not check this too, and a very large pdf file is added (let’s say 250gb and it is copied from a very slow external drive or NAS), it will fail if you want to access that file. But in 99,99% of your cases, pdf files are not big and copied instantaneous.

Yes, that’s what I’m reading here. I’ll certainly take a look. Are you putting the check for write permission in a while loop, BTW? And won’t that fail if the file is read-only (it may never be in your cases, but conceivably could be in mine)?