A neat trick with rsync

Although it's not a widely used feature, rsync allows you to specify multiple source directories along with a single destination directory. rsync essentially overlays all the source directories on top of each other to determine what files to transfer. If a file is in more than one source directory, the first directory the file is found in takes precedence.

Let's say you're managing the /etc directory on a a set of machines, and you're using rsync to make sure the correct files are copied into that directory. Let's further assume that you have multiple directories that can be a source for those files: you might have a Linux directory, a Solaris directory, perhaps an Apache directory if this machine is a web server, and finally a generic directory for files that aren't architecture or application dependent.

On a given destination machine, you have a script which specifies which source directories to rsync from to build /etc. This script will need to choose between the Linux and Solaris directories, and will need to specify the Apache directory if the box is a web server.

The script might start an rsync this way:

  % rsync -e ssh --archive --checksum rserver:/u1/solaris rserver:/u1/apache rserver:/u1/generic /etc

(This example assumes you have a server called rserver.) rsync will first grab any Solaris-specific files, which will override any files in /u1/apache and /u1/generic. Next, the Apache files will be grabbed, followed finally by the generic /etc files. All these files will be copied into /etc on the local machine.

Testing trick

Building on this multiple source directory capability, here's an even better trick. Let's suppose you have a test directory with newer versions of some of the files in the other source directories. If you put this test directory on the front of the rsync source directories list, the test files will override any other files:

  % rsync -e ssh --archive --checksum rserver:/u1/test rserver:/u1/solaris rserver:/u1/apache rserver:/u1/generic /etc

Any files in rserver:/u1/test can override any file in any of the other source directories.

A very sweet trick, indeed!