How to know file is fully copied

General
1

I need a way to know when the user copies a file to a folder, it has been fully copied to that folder.
I tried using a timer and see if the files keeps changing the size. When the size doesn’t get bigger, I assume the file is fully copied to the folder.
Except, this doesn’t always work (especially with larger files).

Any ideas?

BTW macOS only … maybe some sort of declare can help here.

2

what is the source of this file. a user just drag something into “your” folder ?
how about if your app handle this file copy?

3

If you have the MBS plugins, you could use MacFileOperationMBS and check the status of the file operation (it has, among other things, ObjectsComplete, ObjectsRemaining).

4

Don’t know if this would work, but I wonder if you’d get an IOException if you tried to open a binary stream with write permission on the file mid-copy?

Edited for clarity.

5

maybe checking the hash?

https://forum.xojo.com/21340-md5-check-a-file/0

6

Kem’s suggestion is probably easier if it works, failing that, shell out to lsof ? https://ss64.com/osx/lsof.html

The source is here https://opensource.apple.com/source/lsof/lsof-67/lsof/ if you want to go declare digging :slight_smile:

7

[quote=475922:@Markus Rauch]what is the source of this file. a user just drag something into “your” folder ?
how about if your app handle this file copy?[/quote]

Can be from anywhere.

8

[quote=475936:@nicolscanessa]maybe checking the hash?

https://forum.xojo.com/21340-md5-check-a-file/0[/quote]

That’s not possible. The user can copy the file from anywhere. Basically it is not possible to know the original file size.

11

[quote=475934:@Kem Tekinay]Don’t know if this would work, but I wonder if you’d get an IOException if you tried to open a binary stream with write permission on the file mid-copy?

Edited for clarity.[/quote]

Thanks for all the input. Appreciated.
Kem’s solution is good idea. Will try that.

12

FSEventsMBS should tell you when a new file is in a folder that you watch.

13

FS events are great for watching an entire directory but if you want to watch just one file a kqueue might be in order
They are just harder to set up and more resource intensive
https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/FSEvents_ProgGuide/KernelQueues/KernelQueues.html#//apple_ref/doc/uid/TP40005289-CH5-SW2

running lsof on the destination file shold also show which processes are referring to that file
then you can watch those pids to see when they are done

lots of options

14

I am already using fsevents to see when a file is dropped in a folder. But it just gives a trigger a file is added and not when it is completely copied. With large files this can be a problem. That’s why I need a good way to know if the added files is fully copied and available for processing.

15

Will look at kqueue. Thx

16

Can you try to open the file for write access, but not actually write anything? If it is open and being written to (ie open for write access already) this will fail.

17

Instead of waiting until it just doesn’t change… Can you try reading the file size BEFORE you copy, then periodically compare the size of the copied file and once they match you’re done?

18

Curious if my proposal worked…

19

I don’t think opening the file as a stream for writing works …

https://storage.googleapis.com/charlierobin-1245.appspot.com/downloads/file_copy_test.xojo_binary_project.zip

(No exceptions thrown. Even uncommenting the code which writes a string into the file mid-copy doesn’t cause a problem.)

Also tried copying a large file into a directory and running lsof in a terminal whilst the copy was in progress, and didn’t get encouraging results with that either.

20

The check write permission (not with IOexceptions though) solution in a timer is working perfect.
The trick is to use IsWriteable property.

Thanks for this (simple) solution !!

21

I haven’t tried this, but keep in mind that APFS formatting on the Mac will not give you the same results as other OS/format combos if you are copying within the same volume. AFAIK, APFS fakes a copy by creating a hard link behind the scenes, so the “copy” is instant.

The test would be to copy from a different volume.

22

[quote]I haven’t tried this, but keep in mind that APFS formatting on the Mac will not give you the same results as other OS/format combos if you are copying within the same volume. AFAIK, APFS fakes a copy by creating a hard link behind the scenes, so the “copy” is instant.

The test would be to copy from a different volume.[/quote]

There is some other factor involved, as I’m getting different results in my test than the OP is in theirs.

I’ve experimented quickly with test copies of a 2 gig file from APFS to APFS, APFS to Mac OS Extended (Journaled), and Mac OS Extended (Journaled) to APFS. None of them work, either with opening stream/write or by examining IsWriteable (on both old and “new” FolderItem classes).

In all cases, the file being copied becomes available immediately before the copy is complete.

So I would certainly be interested to see how the OP has got it working, as this kind of thing with a “watched folder” kind of scenario is something that has interested me for quite a while.