Recursive on Big folder

  1. 2 weeks ago

    Thomas R

    Jun 25 Europe, France, Besancon

    Two of my applications "SyncTwoFolders" and "ManageNameExt" (on my page ) do recursice call inside folder.
    On big folder (my X-Plane folder contains 176 958 items) I don't have StackOverflow error but it is very long, and I have strange result. I have not the same result in a sub-folder when I treat the big parent folder than when I treat the subfolder only.

    I made a short application which is approximatively the same except that in my app I do something with the items :
    ManyFolderItems.zip
    First column is the index of the items, 2th column is the time it took to dd this item in the ListBox.
    3th column is Ratio, the time it took divide by the index of the item (then the count of items).
    Ratio = 600 x (TicksSinceThreadStarts / LastIndexItems)

    With a folder < 40000 items I don't see problem. On bigger folder (X-plane), Ratio begin at 5 (firsts items) and is 70 for item #7000 and Ratio = 127 for Item #13000 and ratio = 200 at item #22020 .

    I have a MacBook Pro (Retina, 15 pouces, fin 2013) - 2,3 GHz Intel Core i7 - 16 Go 1600 MHz DDR3 under Mojave.

  2. Jean-Yves P

    Jun 25 Testers, Xojo Pro Europe (France, Besançon)

    https://forum.xojo.com/13879-fastest-way-to-get-folderitem-names-into-array/p1#p111418

  3. Thomas R

    Jun 26 Europe, France, Besancon

    Thank you but it's for Window, #If TargetMac is a normal way (it is an old topic then API1).
    I see, and many of my users report me too, that it is much more slower now (API2*) than previous version (API1).

    * : I updated all my applications to API2.

  4. Beatrix W

    Jun 26 Testers, Third Party Store Europe (Germany)

    FileListMBS from the MBS plugin has a parameter not to show the beachball:

    FileListMBS.YieldTicks = 10

    The last time I checked FileListMBS was much faster than the Xojo code. This may have changed for API2. As described here

    https://www.mothsoftware.com/blog_page.php?permalink=chasing-the-beachball-with-instruments-take-two

    FileListMBS isn't perfect. Accessing the folderitems themselves is rather slow. My test was done with 500k files.

  5. Michel B

    Jun 27 Testers, Xojo Pro

    Xojo is rather slow.

    You may want to use the system with a shell.

    See https://helpdesk.kaseya.com/hc/en-gb/articles/229044948-Recursive-directory-listing-in-Windows

    Since shell is not able to do CD, use "&" between commands.

    For instance

    MyShell.Execute("CD Documents & dir /b /s /a:-D > dirlisting.txt")

    Then load the content of dirlisting.txt.

  6. Wayne G

    Jun 27 Testers, Xojo Pro, MVP Auckland, New Zealand

    That's similar to how I do it, however the CD isn't required and you can use the shell.result to read the data directly.

    MyShell.Execute("dir """ + <native path of target folder + """ /b /s /a:-d")
    Var List As String = MyShell.Result

    For very big lists you will probably want to set the timeout to -1 or a large number.

or Sign Up to reply!