Daniel's Blog

Using rsync

There are several ways of getting files from one server to another. One is to use sftp, which is good for a small file or small batch of files, but if you need to sync a large number of files, rsync is the way to go.

Transfer files with limited bandwidth

Command for transferring files between hosts and not wanting to hit the bandwidth or disk too hard. The file's users must have same uid and gid on both sides.

 time rsync -ravzpP --bwlimit=4000 --delete -e "ssh -o StrictHostKeyChecking=no -p <REMOTE_HOST_PORT_NUMBER>" --rsync-path="sudo rsync" /source/directory/on/source/machine/ <REMOTE_HOST_IP_OR_DNS_NAME>:/target/directory/on/destination/machine/

Transfer files ASAP

Command for transferring files between hosts as fast as possible. The file's users must have same uid and gid on both sides.

 time rsync -ravzpP --delete -e "ssh -o StrictHostKeyChecking=no -p <REMOTE_HOST_PORT_NUMBER>" --rsync-path="sudo rsync" /source/directory/on/source/machine/ <REMOTE_HOST_IP_OR_DNS_NAME>:/target/directory/on/destination/machine/

Options Explanation

-r   --recursive      recursivly get the files
-a   --archive        is equavelant to  -rlptgoD 
-v   --verbose        increase verbosity of logging output
-z   --compress       compress the data when transfering 
-p   --perms          maintain permissions
-P                    equvalent to --progress --partial
     --progress       Shows progress
     --partial         Keeps partially transmitted files
-l   --links          compy symlinks as symlinks
-t   --times          preseve modification times
-g   --group          preserve group id
-o   --owner          preserve owner
-D                    same as --devices --specials
     --devices        This option causes rsync to transfer special files such as devices
     --specials       This option causes rsync to transfer special files such as named sockets and fifos
     --bwlimit=##     Limit to ## KBps