CURL (CURLSMBS) FTP : Recursive listing

Hi there!

I try to get recursively the contents of an FTP directory. If I get correctly the contents of the main directory, as soon as I try to open a child-folder, I get the error : «Server denied you to change to the given directory»…

This is my code :

Function loadFiles(folder As String, currentFolder As FTPFolder)
{
if(currentFolder=nil) then
    folder = FTP_mainFolder
    mainFolder = new FTPfolder
    currentFolder = mainFolder
    currentFolder.name = folder
  end if
  
  dim s as string
  dim e as integer
  
  connection = new CURLSMBS
  connection.CollectDebugData = true
  connection.CollectOutputData = true
  connection.OptionVerbose = true
  connection.OptionCustomRequest = "MLSD"
  connection.OptionFreshConnect = true
  connection.OptionUsername = FTP_user
  connection.OptionPassword = FTP_password
  
  if(folder.Left(1)<>"/") then folder = "/"+folder
  if(folder.Right(1)<>"/") then folder = folder + "/"
  dim url As String
  url = "ftp://"+FTP_host+folder
  connection.OptionURL= url
  
  e = connection.Perform
  
  dim results As String = ReplaceLineEndings(connection.OutputData, EndOfLine)
  dim lines() As String = results.Split(EndOfLine)
  dim i As Integer
  
  for i=2 to lines.Ubound
    dim line As new FTPLine
    if(line.parseLine(lines(i))) then
      if(line.Type="file") then
        dim f As new FTPfile
        f.parent = currentFolder
        f.date_modif = line.dateModif
        f.name = line.name
        f.hidden = line.hidden
        currentFolder.files.Append(f)
        
      elseif(line.Type="dir") then
        dim f As new FTPfolder
        f.name = line.name
        f.hidden = line.hidden
        f.parent = currentFolder
        loadFiles(folder+f.name+"/", f)
        currentFolder.folders.Append(f)
        
      end if
    end if
  next
}[/code]

and my classes : 
[code]Class FTPLine
{
  property dateModif As Date
  property hidden As Boolean
  property name As String
  property perm As String
  property size As Integer
  property unique As String

  function parseLine(line As String)
  {
  dim cols() As String = line.Split(";")
  if(cols.Ubound>3) then
    for each col As String In cols
      dim colAttribute As String = col.Left(col.InStr("=")-1)
      dim colValue As String = col.Right(col.len-col.InStr("="))
      select case colAttribute
      case "Type"
        type = colValue
      case "Modify"
        dateModif = new Date
        dateModif.Year = val(colValue.Left(4))
        dateModif.Month = val(colValue.Mid(4,2))
        dateModif.Day = val(colValue.Mid(6,2))
        dateModif.Hour = val(colValue.Mid(8,2))
        dateModif.Minute = val(colValue.Mid(10,2))
        dateModif.Second = val(colValue.mid(12,2))
      case "Perm"
        perm = colValue
      case "Unique"
        unique = colValue
      case "Size"
        size = val(colValue)
      else
        name = colValue
      end select
    next
    
    hidden = (name.Left(1)="." or name.Left(2)=" .")
    
    Return true
  else
    Return false
  end if
  }
}

Class FTPFolder
{
  property files() As FTPFile
  property folders() As FTPFolder
  property hidden As Boolean
  property name As String
  property parent As FTPFolder
}

Class FTPFile
{
  property date_modif As Date
  property hidden As Boolean
  property name As String
  property parent As FTPFolder
}

Any idea?

Add one more “/” slash after the IP…

Part from your code

url = "ftp://"+FTP_host+"folder

to

url = "ftp://"+FTP_host+"//"+folder

something like this

curl “ftp://root:test1@192.168.1.1//var/tmp/test-disk1.dmg”

if for any reason server does not recognize the second "/"slash then add “%2f”.
The “%2f” will be resolved from server as “/”

you got error?
maybe you check debug messages from ftp server?
See DebugMessage event (live) or DebugData property (after transfer).

I already tried the second slash but it was not successful. I tried to replace spaces with %20, it does not change anything.

In fact, as I do not get the result I was waiting for, I added a «stop» in my code to check the state of my CURLSMBS at each step (each folder). For the main directory, no problem, but any child-folder gives me always the same error…

[quote=259054:@SbastienAngot]I already tried the second slash but it was not successful. I tried to replace spaces with %20, it does not change anything.

In fact, as I do not get the result I was waiting for, I added a stop in my code to check the state of my CURLSMBS at each step (each folder). For the main directory, no problem, but any child-folder gives me always the same error…[/quote]
If the error is 9 from your server then is problem with Paths.
Tip.
Escape the slashes by adding another slash before each desired slash
take a look on this maybe help you.
curl ftp

--ftp-method [method] (FTP) Control what method curl should use to reach a file on a FTP(S) server. The method argument should be one of the following alternatives: multicwd curl does a single CWD operation for each path part in the given URL. For deep hierarchies this means very many commands. This is how RFC1738 says it should be done. This is the default but the slowest behavior. nocwd curl does no CWD at all. curl will do SIZE, RETR, STOR etc and give a full path to the server for all these commands. This is the fastest behavior. singlecwd curl does one CWD with the full target directory and then operates on the file "normally" (like in the multicwd case). This is somewhat more standards compliant than 'nocwd' but without the full penalty of 'multicwd'.

I tried

  • multicwd & singlecwd : main folder OK - first child-folder gives «Server denied you to change to the given directory»
  • nocwd : main folder OK - first child-folder gives «RETR response 550»

From what i know the «Server denied you to change to the given directory» have to do with path problems most because the server try to resolve the path as absolute and get relative path.
Can you try to put double “//” on the Home Folder?
«RETR response 550» is Server denied.

I search i little more with my vps server,and i get this results.
All the bellow errors is from proftpd server,if i try with folders without access like,(Home,var,etc,www).
«Server denied you to change to the given directory»
«RETR response 550»
«RETR response 530»
If i try the commands outside the root then working perfect.
One solution is to add Ftp Global userto proftpd server,but is no easy and depends from what panel you using(plesk,cPanel,etc)

ioannis$ curl -o error_log ftp://myName:myPass@myServerIP/logs/error_log % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 13800 100 13800 0 0 13767 0 0:00:01 0:00:01 --:--:-- 13772
The above is from downloaded file,outside the root folders.
For the root access directories best is the rsync command

do you use current plugin?
Did you look into using our CURLSFileInfoMBS class instead of yours?

[quote=259184:@Christian Schmitz]do you use current plugin?
Did you look into using our CURLSFileInfoMBS class instead of yours?[/quote]
Christian i test the example Curls ftp directory listing…
i get the same results when i log as root to list the directory /var/etc from my vps.
Result: 67 = Curle_login_denied
If i log with ftp account and NO root
Result:0 = OK

drwxr-xr-x 2 johnny psacln 4096 Dec 12 15:41 error_docs drwxr-x--- 6 johnny psaserv 4096 Jan 7 22:18 httpdocs drwx------ 3 johnny root 4096 Apr 12 03:49 logs drwxr-xr-x 2 johnny psacln 4096 Dec 13 01:36 odin1 drwxr-x--- 11 johnny psaserv 4096 Dec 17 15:05 orines drwxr-xr-x 2 johnny psacln 4096 Apr 11 09:24 private
The error i get and the error from the OP is from proftpd server…

And this bellow if you have access as ftp user NO global and error 550 because the folder “/var” is not public folder…

220 ProFTPD 1.3.5a Server (ProFTPD) [MyServer] USER johnny 331 Password required for johnny PASS mypass 230 User johnny logged in PWD 257 "/" is the current directory Entry path is '/' EPSV Connect data stream passively ftp_perform ends with SECONDARY: 0 229 Entering Extended Passive Mode (|||20854|) Trying MYIP... Connecting to myIP (myIp) port 20854 Connected to myIP (myIP) port 21 (#0) TYPE I 200 Type set to I SIZE var 550 var: No such file or directory RETR var 550 var: No such file or directory RETR response: 550 Remembering we are in dir ""
I don’t know what paths the @Sébastien Angot use to reproduce the error!!
Any other test i make with public folders the plugins working perfect.