With 242pr4, uploading folders to an SFTP server fails with error 26 - “client read function EOF fail”:
My app first creates the target folder on the server (AWS S3) then uploads files to the folder.
It works fine with MBS 241 and earlier.
That means CURL tries to read data for the transfer, but can’t read.
What does your code look like?
It is possible that we have a bug in the plugin.
Or that you coded it in a way, which didn’t show an error before.
Folders with files to be uploaded are cached on disk in a directory in AppData, and a timer checks that directory periodically to see if anything needs to be uploaded. For each file to upload, I create a CURLSMBS whose URL contains the new target directory corresponding to the file’s parent folder, and set
curl.OptionFTPCreateMissingDirs = 2
so that the target directory will be created on the server if it doesn’t exist.
An uploader thread is instantiated for each file to upload, curl.performMT is called in its Run event, and a reference to each uploader thread is maintained in a dictionary. When the timer checks for pending uploads, it deletes any successfully completed uploaders from the dictionary and their corresponding source files from the cache. Once a cached folder contains no more files, the folder is deleted.
The result is a scheme that will keep retrying in the event of any errors (even an app crash or restart), as source files and folders are not deleted from the disk cache until successfully uploaded. This is critical for this application, where loss of files to be uploaded is disastrous.
Okay, so how do you do the upload?
InputData property? OpenMTInputFile?
You check for error condition like if OpenMTInputFile fails?
I tried earlier today an SFTP upload here and I had no problem so far. Uploaded a couple of files in different ways.
Sample code:
Dim e As Integer
Dim d As New CURLSMBS
Const path = "/Users/cs/Pictures/Test Pictures/IMG_9069.jpeg"
Dim file As New FolderItem(path, FolderItem.PathModes.Native)
Call d.OpenMTInputFile(file)
'd.InputData="Just a little bit text. Have fun!"
d.OptionURL="sftp://ubuntu-linux-22-04-02-desktop.local/home/parallels/Desktop/Folder/test.jpg"
d.OptionUpload=True
d.OptionVerbose = True
d.OptionFTPCreateMissingDirs = 2
d.OptionUsername = "parallels"
d.OptionPassword = "linuxuser"
e=d.Perform
listbox1.addrow "Result: "+str(e)
Dim DebugMessage As String = d.DebugMessages
Dim LasterrorMessage As String = d.LasterrorMessage
Select case e
case d.kError_LOGIN_DENIED
Listbox1.AddRow "Wrong password."
case d.kError_UNSUPPORTED_PROTOCOL
Listbox1.AddRow "Protocol not implemented."
case d.kError_OK
Listbox1.AddRow "No error."
else
break // some other error
end select
Yes, I call OpenMTInputFile and if it fails I don’t instantiate the uploader. It will try again next time the timer fires to check pending uploads, typically at 10-30 second intervals.
For each file as FolderItem in ItemToUpload.Children
DIm d As new UploadCURL
// Sometimes OpenMTInputFile fails, don't know why. But we can just ignore and UploadTimer will try again later
If d.OpenMTInputFile(file) Then
d.OptionURL = EscapedURL(destDir+"/"+ParentFolderName+file.Name)
d.OptionUpload=true
d.OptionConnectionTimeout = ConnectionTimeout
d.OptionUsername =username
d.OptionFTPCreateMissingDirs = 2
d.OptionVerbose = True
d.OptionPort = 22
d.OptionSSHPrivateKeyfile = KeyFile.NativePath
d.OptionSSHAuthTypes = 1 //CURLSSH_AUTH_PUBLICKEY
d.OptionDefaultProtocol = "SFTP"
// Create uploader object and add it to the Uploads dictionary, which is checked periodically in CheckPendingUploads.
// FileUploader.Constructor instantiates a thread in which d.PerformMT is executed.
Uploads.Value(file.NativePath) = new FileUploader(file,destdir,d) // And off it goes
End
Next
Looking again on your log from above, it seems like the connection gets reused.
And the data from the first file is already sent.
I can do multiple uploads here just fine with CURL.
Maybe you forbid reuse of the connection or make it start fresh:
d.OptionForbidReuse = True
or
d.OptionFreshConnect = true
I’ll give that a try, but it’s been working fine literally for years without issue until 242pr4.
Can you reproduce this?
It may be a bug in the newer curl version.
Yes, 100% reproducible with my app under Mac and Windows, on two different machines, on every upload attempt.
I haven’t made a test app, though.
I could reproduce it today
This happens if no input data is set, so the plugin tries to use the last one from the previous upload when the same object is reused. Maybe you find some logic case like this where the upload is built somewhere else?
Every time I upload a folder, a new CURLSMBS is instantiated for each file in the folder. If (and only if) MTInputFile is successfully opened, PerformMT is called, and error 26 results.
So not sure how “no input data” or “same object reused” could be an issue.
Me, too.
No idea how you make it. Sorry, but I can’t reproduce it otherwise.
Please try OptionFreshConnect and if you can make a project to show, let me know.
d.OptionFreshConnect = true