Copying with rsync

Mon, 17 Feb 2020 00:00:00 +0000

Reasons why you should (not always, but often) prefer rsync over cp:

Resume Interrupted Transfers If you are copying a large file and your terminal session crashes or the power cuts out, cp will simply fail, and you have to start over. rsync can pick up right where it left off, which is a massive time-saver for large datasets.
Efficiency (Delta-Transfer Algorithm) rsync is designed to be smart. If you are copying a file that already exists at the destination (e.g., you are updating a backup), rsync only copies the parts of the file that have actually changed (the “delta”) rather than overwriting the entire file. cp, by contrast, always copies the whole file every single time.
Real-time Progress Monitoring rsync provides excellent visual feedback. By using the -P (or –progress) flag, you get a real-time progress bar showing the percentage complete, transfer speed, and estimated time remaining. cp is famously silent, leaving you guessing whether the process is still running or has hung, requiring you to check at the destination via the size of the arriving file, whether anything is progressing (ls -lh filename, or du -hs filename)
Seamless Remote Transfers rsync is natively built to work over SSH. You can use it to copy files directly to or from a remote server as easily as copying files on your local machine. cp cannot do this; to use cp for a remote server, you would first need to mount the remote filesystem locally, which is far more complex and prone to errors.

Here is a common use case, the transfer of a directory, including all of its content:

BackupPC and Macs

Wed, 13 Apr 2016 00:00:00 +0000

I myself have been using the backup software BackupPC for almost a decade, and I started to backup our lab computers to a central backup server about two years ago. BackupPC supports deduplication and therefore much data can be stored on a couple of 2 TB drives.BackupPC supports many protocols (smb, ftp, tar/rsync via ssh), but we mostly use rsync via ssh. The data is not encrypted before the backup. Strangely not even the upcoming version 4 will support pre-egression encryption. However, we store the backup on an encrypted volume. That way it is at least protected if the backup server is stolen. And during transit, the data is protected by ssh. However, the system is not TNO (“trust no-one”), since I (as the backupc administrator) can access the files. When using Linux, one feasible method would be to encrypt the user’s home directory using eCryptFS (which is an inbuilt option when creating users on Ubuntu ) and then backup the /home/.ecryptfs directory instead of the users home directory. However, recovery would be much more of a problem. You would be able to browse the diectory structure and files of the backup, but the filenames would be meaningless since they are also encrypted in the process.There is one peculiarity in backing up Mac OSX machines: BackupPC normally connects as root via ssh into the client computer and executes the rsync backup command. In order to make this possible on university-managed Mac OSX computers, we had to create a dedicated user (“backuppc”) on the client machines and allow for this user the execution of rsync with root privileges, which is done by adding this line in the /etc/sudoers file:backuppc ALL=NOPASSWD: /usr/bin/rsync. Then we have to change the ssh/rsync command for the Mac client on the backuppc server changing “root” into “backuppc”.

Rsync on Michael’s Domain

Copying with rsync

BackupPC and Macs