Recursive on Big folder

Two of my applications “SyncTwoFolders” and “ManageNameExt” (on my page) do recursice call inside folder.
On big folder (my X-Plane folder contains 176 958 items) I don’t have StackOverflow error but it is very long, and I have strange result. I have not the same result in a sub-folder when I treat the big parent folder than when I treat the subfolder only.

I made a short application which is approximatively the same except that in my app I do something with the items :
ManyFolderItems.zip
First column is the index of the items, 2th column is the time it took to dd this item in the ListBox.
3th column is Ratio, the time it took divide by the index of the item (then the count of items).
Ratio = 600 x (TicksSinceThreadStarts / LastIndexItems)

With a folder < 40000 items I don’t see problem. On bigger folder (X-plane), Ratio begin at 5 (firsts items) and is 70 for item #7000 and Ratio = 127 for Item #13000 and ratio = 200 at item #22020 .

I have a MacBook Pro (Retina, 15 pouces, fin 2013) - 2,3 GHz Intel Core i7 - 16 Go 1600 MHz DDR3 under Mojave.

https://forum.xojo.com/13879-fastest-way-to-get-folderitem-names-into-array/p1#p111418

Thank you but it’s for Window, #If TargetMac is a normal way (it is an old topic then API1).
I see, and many of my users report me too, that it is much more slower now (API2*) than previous version (API1).

  • : I updated all my applications to API2.

FileListMBS from the MBS plugin has a parameter not to show the beachball:

FileListMBS.YieldTicks = 10

The last time I checked FileListMBS was much faster than the Xojo code. This may have changed for API2. As described here

https://www.mothsoftware.com/blog_page.php?permalink=chasing-the-beachball-with-instruments-take-two

FileListMBS isn’t perfect. Accessing the folderitems themselves is rather slow. My test was done with 500k files.

Xojo is rather slow.

You may want to use the system with a shell.

See https://helpdesk.kaseya.com/hc/en-gb/articles/229044948-Recursive-directory-listing-in-Windows

Since shell is not able to do CD, use “&” between commands.

For instance

MyShell.Execute("CD Documents & dir /b /s /a:-D > dirlisting.txt")

Then load the content of dirlisting.txt.

That’s similar to how I do it, however the CD isn’t required and you can use the shell.result to read the data directly.

MyShell.Execute("dir """ + <native path of target folder + """ /b /s /a:-d") Var List As String = MyShell.Result

For very big lists you will probably want to set the timeout to -1 or a large number.