Using AuthorizationShell in macoslib with rsync

Hello,

I’m using the AuthorizationShell in macoslib to try to run an rsync command. I realize AuthorizationShell is deprecated, but that’s what I’d like to use, at least for now.

I’ve basically copied the code from the macoslib example, and rsync seems to run, but instead of the normal output, which begins something like this when I run from the terminal:

building file list ... 0 files... 100 files... 114 files to consider

I just get this (and then the command finishes):

building file list ... 0 files to consider

My source folder has hundreds of files in it and my destination folder is empty, so it should definitely see files to sync. It looks like it’s having some sort of problem finding the right folders, at least that’s my guess, since it’s not actually throwing up any error messages. I’m hoping someone can see what I’m doing wrong. This is for copying media files from cameras to a hard drive using checksums to make sure there were no copy errors. It’s based on the command here. I’m just trying to wrap it into a GUI.

I originally had rsync running with no problem in a regular shell class using some of the sudo hacks discussed elsewhere on the forums, but I’d like to use an AuthorizationShell, so I’m not doing anything insecure with the password. One other note, when I was using the built-in shell class, I found that if I used the shellpath for the source and destination, I got an error, an issue with arguments and parsing spaces (even though they were escaped) in filenames. Instead, rsync wanted the nativepath enclosed in quotes. I’ve tried both here (see the commented out args line), and I get the same result either way. Here’s my code:

[code]//source and dest are folderitems to the source and destination folders

dim args() as String
dim rsyncpath as string

dim s as new shell
s.Execute(“which rsync”)
rsyncpath = s.Result.Trim

AuthorizationShell1.Reset()
AuthorizationShell1.SetUsernameAndPassword( “”, “” ) // Even if it’s blank

//args = Array("-rt","-vi","–progress","–exclude="".""",source.shellpath+"/",dest.shellpath)
args = Array("-rt","-vi","–progress","–exclude="".
""",""""+source.nativepath+"/""",""""+dest.NativePath+"""")

if AuthorizationShell1.AuthorizeAndExecute(rsyncpath, args) then
// Shell is running now.
// Let’s start the poll timer and enable command input.
BringSelfToFront
elseif AuthorizationShell1.ErrorNumber <> 0 then
msgBox "Unknown error: " + str(AuthorizationShell1.ErrorNumber)
else
//msgBox “User cancelled”
end if[/code]

In the AuthorizationShell DataAvailable event, I just ReadAll and output it to a textarea, which is where I’m getting the output at the top of this post.

Any help would be appreciated if someone sees what I’m doing wrong. Thanks!

I’m not sure what the issue is, but if “which rsync” works, you don’t need the full path to rsync. Make sure you’re using the version you want to be using by setting the path to rsync yourself.

What do you get if you output this to your TextArea?

Join( args, EndOfLine )

Do the arguments look correct? What if you use rsyncpath + Join( args, " " ), then copy and paste that into the Terminal. Does it work?

BTW, it looks like I have to update the comments in the example project to remove the reference to the “timer”.

Thanks, Kem. I’ll check the args thing in a minute, that’s a good call. But I included the which command because if I just used “rsync” as the command in AuthorizationShell, I got a -60031 error, which is “The tool failed to execute.”

Right, you’re supposed to include the full path, just make sure it’s the same version that you’re using the Terminal.

Gotcha. Yes, I double checked the version, it’s the same (2.6.9).

Here’s the argument list:

-rt -vi --progress --exclude=".*" "/Volumes/NO NAME/" "/Volumes/Sync Test Volume/Cards/Card 1"

And here’s the full command with the arguments:

/usr/bin/rsync -rt -vi --progress --exclude=".*" "/Volumes/NO NAME/" "/Volumes/Sync Test Volume/Cards/Card 1"

Interestingly, when I run this in the terminal, I get a little more information:

[code]/usr/bin/rsync -rt -vi --progress --exclude=".*" “/Volumes/NO NAME/” “/Volumes/Sync Test Volume/Cards/Card 1”
building file list …
rsync: link_stat “/Volumes/NO NAME/.” failed: Invalid argument (22)
0 files to consider

sent 21 bytes received 20 bytes 82.00 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9][/code]

If I run the exact same command in the terminal with “sudo” in front of it and then enter my password, it works perfectly.

Is the AuthorizationShell asking for your username and password when you try to run it? Because you are using SetUsernameAndPassword with blank data and, if memory serves, it will try to use exactly that in an attempt to authorize.

Comment that line and see what happens.

Thanks, yes, it does ask. And it looks like commenting out the SetUsernameAndPassword line makes no difference.

Well, I am at a loss. I can reproduce your results exactly, but if I comment your active args = … line and uncomment the one above it (using ShellPath), it works perfectly.

Thanks for all your help, Kem, this is above and beyond. You got my hopes up there! I thought maybe I hadn’t tested the ShellPath line since fixing some other bug, but no such luck, I still get the same problem. Any chance you could paste in your method to see if I’m missing anything? I’m also on 10.9.1, could Apple have removed it in an update?

One other interesting test: these are files that come off a camera, so the source drive is formatted FAT-32 and sometimes has other weird file locking or permissions issues. I just tried using a source folder on a normal Mac OS Extended (Journaled) drive and with that, it recognized that there were files to sync:

building file list ... 0 files... 2 files to consider

But it didn’t actually sync the files, it just went straight to the “Completed” event. Pasting the same command into the terminal, this time without sudo, worked fine. This is all using the shellpath args, it still finds “0 files to consider” with nativepath in Xojo. Although either method works in the terminal without sudo for the Mac OS formatted source drive.

D’OH! I always forget that OS calls (which is what is behind AuthorizationShell) always take the NativePath, not the ShellPath, so try this and see what happens:

  args = Array("-rt","-vi","--progress","--exclude="".*""",source.NativePath+"/",dest.NativePath)

Well, something different happened, anyway. It didn’t see any files, but the output changed slightly:

[code]building file list …
0 files to consider

sent 21 bytes received 20 bytes 82.00 bytes/sec
total size is 0 speedup is 0.00[/code]

Running it in terminal, it looks like the paths need to be enclosed in quotes:

[code]building file list …
rsync: link_stat “/Volumes/NO” failed: No such file or directory (2)
rsync: link_stat “/Users/davidheidelberger/NAME/.” failed: No such file or directory (2)
rsync: link_stat “/Volumes/Sync” failed: No such file or directory (2)
rsync: link_stat “/Users/davidheidelberger/Test” failed: No such file or directory (2)
rsync: link_stat “/Users/davidheidelberger/Volume/Cards/Card” failed: No such file or directory (2)
0 files to consider

sent 21 bytes received 20 bytes 82.00 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9][/code]

But enclosing them in quotes is identical to the original args assignment I had in the code.

Right, the path should not be in quotes for the OS call, and it should be the NativePath.

OK, now I have to see if it’s a permissions thing…

Well, I create a MS-DOS formatted disk using a disk image and didn’t have any trouble syncing the folder. Can you post your new code with the updated args = …?

Sure, thanks.

Here’s my method, which is in the action event of a pushbutton:

[code]//source and dest are folderitems to the source and destination folders

dim args() as String
dim rsyncpath as string

rsyncpath = “/usr/bin/rsync”

AuthorizationShell1.Reset()

//Neither of these work
//args = Array("-rt","-vi", “–progress”, “–exclude=”".""", “”""+source.NativePath+"/""", “”""+dest.NativePath+"""")
args = Array("-rt", “-vi”, “–progress”, “–exclude=”".
""", source.NativePath+"/", dest.NativePath)

outputText.Text = join(args,EndOfLine)
outputText.AppendText EndOfLine+rsyncpath+" “+join(args,” ")+EndOfLine+EndOfLine

if AuthorizationShell1.AuthorizeAndExecute(rsyncpath, args) then
// Shell is running now.
// Let’s start the poll timer and enable command input.
//pgFiles.Visible = true
//pgThisFile.Visible = true
//lblFiles.Visible = true
//lblThisFile.Visible = true
BringSelfToFront
elseif AuthorizationShell1.ErrorNumber <> 0 then
msgBox "Unknown error: " + str(AuthorizationShell1.ErrorNumber)
else
//msgBox “User cancelled”
end if[/code]

This is my code in the AuthorizationShell DataAvailable event:

outputtext.AppendText me.ReadAll outputtext.ScrollPosition = Len(outputtext.Text) ' scroll to the end

And in the Completed event:

[code] 'pgFiles.Visible = false
'pgThisFile.Visible = false
'lblFiles.Visible = false
'lblThisFile.Visible =false

outputText.AppendText “Done”[/code]

This is the output in the textarea:

[code]-rt
-vi
–progress
–exclude="."
/Volumes/NO NAME/
/Volumes/Sync Test Volume/Cards/Card 1
/usr/bin/rsync -rt -vi --progress --exclude=".
" /Volumes/NO NAME/ /Volumes/Sync Test Volume/Cards/Card 1

building file list …
0 files to consider

sent 21 bytes received 20 bytes 82.00 bytes/sec
total size is 0 speedup is 0.00
Done[/code]

This is a link to a zip with some of the contents of the drive I’m trying to copy. You’ll see that the files are locked. I don’t know if that has anything to do with it, but again, it works in Terminal with sudo.

No, the contents of the source shouldn’t matter since your are copying from that folder, not to it.

Not to bypass the issue, but this won’t work with an ordinary Shell?

I did get it to work in an ordinary Shell using “sudo rsync…”, but reading this thread made it look like there are security issues with doing it that way.

That’s true, but you said:

If it works without sudo, it should work with the straight Shell, no?

Ah. Good point. But that only seemed to work when the source was on the Mac OS formatted drive. For some reason, either because of the FAT-32 or the locked files, or some combination, it seems like sudo is required for these camera files.

Oh, so it won’t work from the FAT-32 in Terminal unless you use sudo? Please confirm that, and I’ll continue trying to help with AuthorizedShell.

I’m not sure if it’s FAT-32 or if it’s some other factor thats causing that, but I’ll try to test a few things.