rsync is a UNIX command to synchronize files and folders between two computers. It is very useful to download files to and from the server. The key difference to more commonly used scp (SSH copy) is that rsync can do incremental copies: it only copies the modified files. This is especially useful if you sync a large source code tree between different computers. rsync should be in any developer’s toolbox who are working with UNIX based software and deployments.

rsync works over SSH by default (I think it was over telnet by default in long past, but nowadays it’s SSH based, someone please refresh my memory). rsync also supports SSH:s on-line compression, which greatly reduces bytes needed to transfer when copying textual content (source code, logs).

To copy a remote folder to your local computer:

rsync –compress-level=9 -av username@server:/home/user/yourlogfiles

Explanation

• –compression-level: Use maximum on-line compression
• -a: Archive. Keep file timestamps in intact and copy only modified files.
• -v: Verbose. Show what files we are copying.

If the copy process is interrupted rsync will resume the copy from the last file it didn’t manage to copy. In the end rsync tells the copy speed and (incremental) speed-up stats:

sent 376 bytes  received 80696946 bytes  658753.65 bytes/sec
total size is 2604253766  speedup is 32.2

Note: incremental copy might be confused by continuously changing files (logs, databases).

See also a prior blog post about scp copy and compression level.

 Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+