File I/O speed differences (OS X / Windows 8.1)

The multimedia project I am working on loads the contents of a main folder (who holds 972 folders), then it reads in a loop if a child have a certain file and if it have the right file inside, it store the folder name into a ListBox.

The time spent for the same operations on OS X is… instantaneous (or less than a second) on OS X, but around 10 minutes on Windwos 8.1.

The data are the same: the applications and data are stored in a SD HC card (not so fast, not xxxx 4, boot on this SD Card is slow).

The application are run from the SD Card (both applications), no other running applications at the same time.

For the record: here is the structure of the main folder:

<Main Folder> <Child Folder 0> <file text 1 [mandatory]> <file text 2> <file image 1> <file image eventually #2, but more image files can be found>

Once the program gets the main folder (by a drop, usually),
the program runs a loop,
if the Child if this is a folder,
the progran search about a mandatory text file
if found, it adds this Child folder name into the ListBox
Else, it reads the next Child folder

BTW: there is one folder exception to the process above: one service data folder with… service files. If one or more of these mandatory items is missing, the process stops. A second folder exists, but its contents is analyzed only on a different condition.

a. there is no ListBox addition until the whole process is done, (I do not care because on OS X it was very fast and changes to that means more time to launch)

b. in Windows 8.1 the blue circle runs during the read process,

c. In Windows 8.1 I can run an application (after some minutes, I considered that the process run time is not that important; the initialization process is too slow),

d. In Windows 8.1 I cannot open / close / move a folder in the Explorer.

c. The project have been created using Xojo, it is not an evolution of a project created with REALbasic or Real Studio (this seems to matters).

Now is the time to make a time testing project specifically for Windows 8.1.

Results:

  1. Now Both the test project and the original application are slow (on OS X)!
    What happens ? Excepted maybe a memory lack ?

  2. I found a bunch of bugs:
    a. Create a new File Type,
    Add a file type
    In the File Type Name Field, type something, then press Return, more than once
    and watch what happens…
    b. The following compiles:
    Me.AcceptFileDrop “special/folder”

    c. And if I add:
    TF_Start.Text = Obj.FolderItem.Name

     I get the name of the dropped object (even if it is not a folder…
    
     same apply with:
     Me.AcceptFileDrop FT.All // Defined as "special/folder"
    
     Remember: the docs said (once upon a long ago) that the passed file type is for filtering the dropped item (!)
    

Item c above is known and said as “not a bug”.

  1. Difference in code explain (why ?) the difference of time excecution: ?
    The original is slow:
    If ChildFI.Child("Comments.txt") <> Nil Then

    The new code is fast (but not enoug on Windows 8.1)
    If ChildFI.Child("Comments.txt").Exists Then

Note: the original code below returned an error and so I changed it to discover the double dot…
Also: I had to change that code because it was slow (but I do not knew it was that code) and because that code reported every found item (thanks to the bug in 2.c:

     [code]If ChildFI..Child("Comments.txt").Exists Then[/code]

Running specs:

OS X: 70 ticks (5 ticks from the IDE) and 974 items checked (LoopIdx value).
Windows 8.1: 3151 (7523 last code ! *) ticks and 1946 items ** checked (LoopIdx value).

Further testing (adding a leading ChildFI.Visible And show a time increase…)

The only test I do not do is to split the one line If into one If line for a test (that would explain why I originally got a difference in the running time and why the running time doubled once I added the (useless ?) ChildFI.Visible check in the If block.

BTW: I know, running an application and using it from a SD Card is not recommanded (and so is doig the same with a CD / DVD / Blu-Ray…), but you know what users can do.

  • same kind of timing after a power off … wait … power on and un only this testing application… on WIndows 8.1.

** The difference in the number of item checked is explained by the fact the original master folder comes from OS X and then holds OS X invisibles service files.
I the original folders, all folders have a custom icon (icon+cr named item), have specific bounds (.DS_Store item), etc.

And for those who want to know “What the heck does it do to be so slow”, here’s the code:
(I removed the comments lines)

[code]Sub DropObject(obj As DragItem, action As Integer)
TF_Start.Text = Obj.FolderItem.Name // Display a bug… [drop a file on the ListBox]

If Obj.FolderItemAvailable Then
Dim ChildCnt As Integer
Dim LoopIdx As Integer
Dim BookFI As FolderItem
Dim ChildFI As FolderItem
Dim TicksStart As Integer
Dim TicksEnd As Integer

Me.DeleteAllRows
TicksStart = Ticks
BookFI = Obj.FolderItem
ChildCnt = BookFI.Count

If ChildCnt < 3 Then Exit // 2 mandatory folders… plus one image, eventually…

For LoopIdx = 1 to ChildCnt     // Scan the BookFI folder contents
  ChildFI = BookFI.Item(LoopIdx)      // Get a Child
  
  If ChildFI.Visible And ChildFI.Directory And ChildFI.Child("Comments.txt").Exists Then
    LB.InsertRow(LB.ListCount,ChildFI.Name)
  End If
  
  If UserCancelled Then Exit
Next

TF_Items.Text = "LoopIdx: " + Str(LoopIdx)
 TicksEnd = Ticks
 TF_End.Text = Str(TicksEnd - TicksStart)

End If
End Sub[/code]

I do not think that the ChildFI.Visible test is useful (but it slow down the whole process). Also, I am searching a text file inside the ChildFI folder… Clearly (IMHO), checking for invisibles is useless.

When I will be back near to a Windows 8.1 laptop, I will test:

If ChildFI.Directory Then // Is this child of the master folder is a Directory ? If ChildFI.Child("Comments.txt").Exists Then // Yes ! And to be included, it must be named… "Comments.txt" LB.InsertRow(LB.ListCount,ChildFI.Name) End If End If

just in case it can be faster…

why not make a method ?

Sub ScanFiles(file as folderItem)
  if file.directory then
    for i as integer = 1 to file.count
      ScanFiles file.item(i)
    next
  else
    if file.Name = "Comments.txt" then
      LB.AddRow
      LB.Cell(LB.LastIndex, 0) = file.DisplayName
    end
  end if
End Sub

and in DropObject

  if obj.FolderItemAvailable then
    me.DeleteAllRows
    ScanFiles obj.FolderItem
  end if

Axel:

and you think this will speed-up things whie running on Windows 8.1 ?

If you want to plough through files quickly I strongly recommend the calls in Karen Atkocius’ post in this thread:

https://forum.xojo.com/13879-fastest-way-to-get-folderitem-names-into-array/0

Like greased lightning.

Thank you for the link, Peter.

Axel:

Thank you for the provided code.

I tested that code (far above) and it is by far slowest even with some small changes:

I’ve changed your two lines by this single (faster by 3 to 4 times; I saw a 80,000 Ticks run but I do not noted the read ellapsed # of ticks and due to its length, I do not wanted to run it…):

LB.AddRow file.Parent.Name // DisplayName is meant to be used in that case

Also, as you can see, I report in the ListBox the parent of the text file (that is what I want and what I do in my own code).

At last, I forgor to replace your .AddRow (said in the docs to be slower thatn .InsertRow), but the results are:

122 ticks on OS X (vs 70) and far, far more on Windwos 8.1 (20,380 and really far… more using your original code).

Oh, I added the code to report the running Ticks (outside of your process code)

BTW: I do not had time to test the WIndows Declare code.

In OSX I see no big difference between AddRow and InsertRow

I get
AddRow - 19807 Files / 300 Ticks
InsertRow - 19807 Files / 299 Ticks

[quote=190133:@Axel Schneider]In OSX I see no big difference between AddRow and InsertRow

I get
AddRow - 19807 Files / 300 Ticks
InsertRow - 19807 Files / 299 Ticks[/quote]
I cannot comment, I only noticed that minutes ago and do not have my Windows 8.1 machine.

But for 1 tick… (on 5 minutes…)