Read contents of directory

Thomas_Tempelmann · March 20, 2017, 9:51am

Uh, Beatrix, I shortened my post right when you posted your reply, so it seems out of context now, maybe you want to delete that again.

Neil_Burkholder · March 20, 2017, 12:41pm

I’m amazed and impressed by the list of plugins @Christian Schmitz has made. I think it was appropriate to list the plugin, as it is a valid solution. I would do the same if I had spent the time to make a plugin.

If I wanted a the quickest solution to get an app out this would definitely be the choice. On the other hand, people regularly invest large amounts of time and money for education. I started using Xojo about two years ago, and I’m finally to the point that sales have paid for all the time and money I spent learning Xojo.

In this particular case I would like to brush up on my limited experience with declares. I’ve done enough of declares to understand the concept, but structures and data types are still somewhat of a mystery to me.

While Christians plugin is an answer I’m not marking it as the answer because in this case it was not the answer I was looking for.

To reiterate, here is what I wish to accomplish using declares rather than folder items for the sake of speed.

Write a function that takes a path (string), and dictionary as arguments and recurses through all subfolders to return the dictionary with full path as the key and file name as the value.
At this point I’m interested in a windows only solution.

Here is my working VB Net solution that I’m trying to mimic.


    ListBox1.Items.Clear()
    Dim files() As String
    Dim path As String = "C:\\Users\\Server Computer\\OneDrive\\Job Proposals"
    ListBox1.Items.AddRange(GetFileNames(path, files))

    Function GetFileNames(path As String, files() As String) As String()
        If files Is Nothing Then files = {}
        Dim f1() As String
        f1 = IO.Directory.GetFiles(path)
        Dim d1() As String = IO.Directory.GetDirectories(path)
        For Each d As String In d1
            f1 = GetFileNames(d, f1)
        Next
        f1 = files.Union(f1).ToArray
        Return f1
    End Function

So if there is someone out there with Win API declares experience your input would be appreciated.

Thomas_Tempelmann · March 20, 2017, 1:17pm

First, I googled “winapi list folder”. That got me to
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365200(v=vs.85).aspx
That shows that you’d use FindFirstFile, FindNextFile and FindClose. Click them to see their declarations.

The first is declared as:

HANDLE WINAPI FindFirstFile( _In_ LPCTSTR lpFileName, _Out_ LPWIN32_FIND_DATA lpFindFileData );

Now translate those types:

HANDLE -> Ptr or Integer
LPCTSTR -> CString (I think, there may be a more suitable Win-compliant String type)
LPWIN32_FIND_DATA -> That appears to be a struct, which you can access in Xojo either as a Structure (Type: the Structure’s name) or a MemoryBlock (-> Type: Ptr)

Let’s try with a Structure. So, declare one based on this: https://msdn.microsoft.com/en-us/library/windows/desktop/aa365740(v=vs.85).aspx

Types:

DWORD -> UInt32
FILETIME -> UInt64 (it’s a struct of two DWORDs as you can see when you follow the link)
TCHAR[count] -> String*count

That leaves the question what MAX_PATH is. After some googling I believe it’s 260.

Name the new Structure LPWIN32_FIND_DATA.

That leads us to the declare for the function:

declare function FindFirstFile lib “user32.dll” (name as CString, ByRef data as LPWIN32_FIND_DATA) as Ptr

That should be a start for going on with the other function declares.

Neil_Burkholder · March 20, 2017, 1:29pm

I tried the example posted here.
It uses declares to walk through a folder. What I couldn’t figure out is how to tell if the returned items are folders or files. Note that even the DOS folders ‘…’ and ‘.’ are returned. WOW.

Neil_Burkholder · March 20, 2017, 1:47pm

OK I think I have it figured out. I’ll post back the complete method once I have it finished.

Thanks everyone.

Neil_Burkholder · March 20, 2017, 4:18pm

OK can anyone tell me what I’m doing wrong? I can step through the code and it works fine. Occasionally I can run it OK, but most of the time my app crashes.

  Listbox1.DeleteAllRows
  dim files() as string
  files = GetFiles("C:\\Users\\Server Computer\\OneDrive\\Job Proposals")
  if  files.Ubound > -1 Then
    for i as integer = 0 to files.Ubound
      Listbox1.AddRow("", files(i))
    next
  end if
 

Function GetFiles(Path As String, SearchPattern As String = "*") As String()
  dim files(), Dir() as String
  if path.Right(1) <> "\" Then Path = path + "\"
  Declare Function FindFirstFile Lib "Kernel32" Alias "FindFirstFileW" (FileName As WString, FindData As Ptr) As Integer
  Declare Function FindNextFile Lib "Kernel32" Alias "FindNextFileW" (FindHandle As Integer, FindData As Ptr) As Boolean
  Declare Function FindClose Lib "Kernel32" (FindHandle As Integer) As Boolean
  
  Dim Result As New MemoryBlock(318)  //A WIN32_FIND_DATA structure
  Dim FindHandle As Integer = FindFirstFile("//?/" + ReplaceAll(path, "/", "//") + SearchPattern + Chr(0), Result)
  
  
  'walk through folder and get list of directories and files
  If FindHandle > 0 Then
    Do
      dim t as Int32
      t = result.int32value(0)
      if t = 16 Then
        if Result.WString(44) <> ".." and Result.WString(44) <> "." Then
          dir.Append(path  + Result.WString(44) + "\")
        end if
      else
        files.Append(path  + Result.WString(44))
      end if
    Loop Until Not FindNextFile(FindHandle, Result)
    Call FindClose(FindHandle)
  End If
  
  'recurse through subdirectories
  if dir.Ubound > -1 Then
    dim d() as String 
    for i as integer = 0 to dir.Ubound
      d = GetFiles(dir(i),SearchPattern)
      if d.Ubound > -1 Then
        for j as integer = 0 to d.Ubound
          Files.Append d(j)
        next
      end if
    next
  end if
  
  Return files
End Function

Neil_Burkholder · March 20, 2017, 4:42pm

I’m please with the read time 3.78 milliseconds to read 1077 files. The crashes are driving me bonkers though. Any ideas? I can’t post a link to a sample project if it makes it easier, although it’s really just a window with a button and a listbox, with the above code.

Andrew_Lambert · March 20, 2017, 5:00pm

How is it crashing, and where?

Neil_Burkholder · March 20, 2017, 5:21pm

DebugMyApplication.exe has stopped working…

Using visual studio debuger i can get this ‘Exception thrown at 0x0F4D1AFA (XojoGUIFramework32.dll) in DebugMyApplication.exe: 0xC0000005: Access violation writing location 0x00490052’

It’s crashing in the recursion and it’s hit and miss. It seems like the crash occurs about the 2nd subfolder recursion. There is no rhyme or reason. As I said if I step through the code the app never crashes.

Neil_Burkholder · March 20, 2017, 5:46pm

Strangely it seems to be crashing at this line:

dim t as Int32

Update:

Fixed that by putting dim at top.

Now it crashes (sometimes) at the files.append(path + Result.WString(44))

I tried changing to:

        temp = path + Result.WString(44)
        files.Append(temp)

But it’s crashing at the append.

Neil_Burkholder · March 20, 2017, 6:04pm

It’s always crashing when appending the last file from the subfolder. Hmm… strange.

Emile_Schwarz · March 20, 2017, 6:48pm

Do you know that Xojo have a debugger ?

Neil_Burkholder · March 20, 2017, 7:55pm

Yeah but occasionally (while using the Xojo debugger) the app will crash without throwing any exceptions (‘…exe’ has stopped working’). If VS is install you have the option to debug the compiled app in VS. Usually not much information of value is available there, but you can see the stack and often the name of the dll where the exception occurred. Pretty low level stuff. Ugh

Notice I’m using counters and label.refresh so I can monitor where in the code the crash is occurring.

anon20074439 · March 20, 2017, 8:09pm

You are increasing the size limit of MAX_PATH by prepending “\\?” so your memory block isn’t big enough.

Change the MemoryBlock(328) to MemoryBlock(100000) as a hack, you will need to figure out this size though (I’ve not read much into it)

PS. I dont know why you are using the / instead of \ and you dont need to escape \ so I see no need for \\

Also, change Return files to Return files() at the bottom of GetFiles just to be sure.

Works here 100% now with that MemoryBlock correction.

Neil_Burkholder · March 20, 2017, 8:15pm

Thank you very much that fixed the issue. Learning is painful.

anon20074439 · March 20, 2017, 8:26pm

No problemo.

Oh there you go…

MAX_PATH is 32,767 if using “\\?” I’ll let you do the math with the rest of the struct

Neil_Burkholder · March 20, 2017, 8:30pm

Thanks to everyone. This is about 4 times faster than the VB Net method GetFiles(path)

Here is the working code:


dim files() as string = GetFiles("C:\\Users\\Server Computer\\OneDrive\\Job Proposals")
  

Function GetFiles(Path As String) As String()
  dim files(), Dir() as String
  if path.Right(1) <> "\" Then Path = path + "\"
  Declare Function FindFirstFile Lib "Kernel32" Alias "FindFirstFileW" (FileName As WString, FindData As Ptr) As Integer
  Declare Function FindNextFile Lib "Kernel32" Alias "FindNextFileW" (FindHandle As Integer, FindData As Ptr) As Boolean
  Declare Function FindClose Lib "Kernel32" (FindHandle As Integer) As Boolean
  
  Dim Result As New MemoryBlock(40000)  //A WIN32_FIND_DATA structure
  Dim FindHandle As Integer = FindFirstFile("\\\\?\" + path + "*" + Chr(0), Result)
  
  'walk through folder and get list of directories and files
  If FindHandle > 0 Then
    Do
      if result.int32value(0) = 16 Then
        'if this is a directory then resurse through the directory to get all files
        if Result.WString(44) <> ".." and Result.WString(44) <> "." Then
          dim d() as String
          d = GetFiles(path  + Result.WString(44) + "\")
          if d.Ubound > -1 Then
            for j as integer = 0 to d.Ubound
              Files.Append d(j)
            next
          end if
        end if
      else
        files.Append(path + Result.WString(44))
      end if
    Loop Until Not FindNextFile(FindHandle, Result)
    Call FindClose(FindHandle)
  End If
  
  Return files()
End Function

anon20074439 · March 20, 2017, 8:35pm

Use 10000 if you want some future random crash / memory corruption

I’d personally go with 40000 just to be safe if you dont want to work out the struct size yourself

32,767 is an approximation as it can be expanded at runtime.

That whole section seems a little vague.

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx

And put the \\?\ the right way round you’re triggering my ocd its only working because luckily one of those calls changes / to \

Neil_Burkholder · March 20, 2017, 8:44pm

Thanks for the explanations. I never would have figured it out on my own. I edited the code in my last post.

anon20074439 · March 20, 2017, 8:45pm

Nice to see it working.

Oh, shameless self promotion if you play around with Declares in the future

http://blog./2017/01/22/windows-to-xojo-data-type-conversion/