15+ rsync Command Examples in Linux

Description :

rsync is a program that behaves in much the same way that rcp does, but has many more options and uses the rsync remote-update protocol to greatly speed up file transfers when the destination file already exists.

The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network link, using an efficient checksum-search algorithm described in the technical report that accompanies this package.

Some of the additional features of rsync are:

1. Support for copying links, devices, owners, groups and permissions
2. Exclude and exclude-from options similar to GNU tar
3. A CVS exclude mode for ignoring the same files that CVS would ignore
4. Can use any transparent remote shell, including rsh or ssh
5. Does not require root privileges
6. Pipelining of file transfers to minimize latency costs
7. Support for anonymous or authenticated rsync servers (ideal for mirroring)

Usage :

rsync [OPTION]… SRC [SRC]… [USER@]HOST:DEST

rsync [OPTION]… [USER@]HOST:SRC DEST

rsync [OPTION]… SRC [SRC]… DEST

rsync [OPTION]… [USER@]HOST::SRC [DEST]

rsync [OPTION]… SRC [SRC]… [USER@]HOST::DEST

rsync [OPTION]… rsync://[USER@]HOST[:PORT]/SRC [DEST]

Options :

-v, –verbose
increase verbosity
-q, –quiet
decrease verbosity
-c, –checksum
always checksum
-a, –archive
archive mode
-r, –recursive
recurse into directories
-R, –relative
use relative path names
-b, –backup
make backups (default ~ suffix)
–backup-dir
make backups into this directory
–suffix=SUFFIX
override backup suffix
-u, –update
update only (don’t overwrite newer files)
-l, –links
copy symlinks as symlinks
-L, –copy-links
copy the referent of symlinks
–copy-unsafe-links
copy links outside the source tree
–safe-links
ignore links outside the destination tree
-H, –hard-links
preserve hard links
-p, –perms
preserve permissions
-o, –owner
preserve owner (root only)
-g, –group
preserve group
-D, –devices
preserve devices (root only)
-t, –times
preserve times
-S, –sparse
handle sparse files efficiently
-n, –dry-run
show what would have been transferred
-W, –whole-file
copy whole files, no incremental checks
–no-whole-file
turn off –whole-file
-x, –one-file-system
don’t cross filesystem boundaries
-B, –block-size=SIZE
checksum blocking size (default 700)
-e, –rsh=COMMAND
specify rsh replacement
–rsync-path=PATH
specify path to rsync on the remote machine
-C, –cvs-exclude
auto ignore files in the same way CVS does
–existing
only update files that already exist
–ignore-existing
ignore files that already exist on the receiving side
–delete
delete files that don’t exist on the sending side
–delete-excluded
also delete excluded files on the receiving side
–delete-after
delete after transferring, not before
–ignore-errors
delete even if there are IO errors
–max-delete=NUM
don’t delete more than NUM files
–partial
keep partially transferred files
–force
force deletion of directories even if not empty
–numeric-ids
don’t map uid/gid values by user/group name
–timeout=TIME
set IO timeout in seconds
-I, –ignore-times
don’t exclude files that match length and time
–size-only
only use file size when determining if a file should be transferred
–modify-window=NUM
Timestamp window (seconds) for file match (default=0)
-T –temp-dir=DIR
create temporary files in directory DIR
–compare-dest=DIR
also compare destination files relative to DIR
-P
equivalent to –partial –progress
-z, –compress
compress file data
–exclude=PATTERN
exclude files matching PATTERN
–exclude-from=FILE
exclude patterns listed in FILE
–include=PATTERN
don’t exclude files matching PATTERN
–include-from=FILE
don’t exclude patterns listed in FILE
–version
print version number
–daemon
run as a rsync daemon
–no-detach
do not detach from the parent
–address=ADDRESS
bind to the specified address
–config=FILE
specify alternate rsyncd.conf file
–port=PORT
specify alternate rsyncd port number
–blocking-io
use blocking IO for the remote shell
–no-blocking-io
turn off –blocking-io
–stats
give some file transfer stats
–progress
show progress during transfer
–log-format=FORMAT
log file transfers using specified format
–password-file=FILE
get password from FILE
–bwlimit=KBPS
limit I/O bandwidth, KBytes per second
–read-batch=PREFIX
read batch fileset starting with PREFIX
–write-batch=PREFIX
write batch fileset starting with PREFIX
-h, –help
show this help screen

Examples :

1. Synchronize Two Directories in a Local Server

To sync two directories in a local computer, use the following rsync -zvr command.

$ rsync -zvr /var/opt/installation/inventory/ /root/temp
building file list ... done
sva.xml
svB.xml
.
sent 26385 bytes  received 1098 bytes  54966.00 bytes/sec
total size is 44867  speedup is 1.63
 
$ ls -l /var/opt/installation/inventory/sva.xml /root/temp/sva.xml
-r--r--r-- 1 bin  bin  949 Jun 18  2009 /var/opt/installation/inventory/sva.xml
-r--r--r-- 1 root bin  949 Sep  2  2009 /root/temp/sva.xml

Note: rsync doesnt preserve timestamps on sync.

2. Preserve timestamps during Sync using rsync -a

rsync option -a indicates archive mode.

$ rsync -azv /var/opt/installation/inventory/ /root/temp/
building file list ... done
./
sva.xml
svB.xml
.
sent 26499 bytes  received 1104 bytes  55206.00 bytes/sec
total size is 44867  speedup is 1.63
 
$ ls -l /var/opt/installation/inventory/sva.xml /root/temp/sva.xml
-r--r--r-- 1 root  bin  949 Jun 18  2009 /var/opt/installation/inventory/sva.xml
-r--r--r-- 1 root  bin  949 Jun 18  2009 /root/temp/sva.xml

As you see above, rsync preserved timestamps during sync.

3. Synchronize Only One File

$ rsync -v /var/lib/rpm/Pubkeys /root/temp/
Pubkeys
 
sent 42 bytes  received 12380 bytes  3549.14 bytes/sec
total size is 12288  speedup is 0.99

4. Synchronize Files From Local to Remote

$ rsync -avz /root/temp/ root@192.168.200.10:/home/abc/temp/
Password:
building file list ... done
./
rpm/
rpm/Basenames
rpm/Conflictname
 
sent 15810261 bytes  received 412 bytes  2432411.23 bytes/sec
total size is 45305958  speedup is 2.87

While doing synchronization with the remote server, you need to specify username and ip-address of the remote server. You should also specify the destination directory on the remote server. The format is username@machinename:path

5. Synchronize Files From Remote to Local

When you want to synchronize files from remote to local, specify remote path in source and local path in target as shown below.

$ rsync -avz root@192.168.200.10:/var/lib/rpm /root/temp
Password:
receiving file list ... done
rpm/
rpm/Basenames
.
sent 406 bytes  received 15810230 bytes  2432405.54 bytes/sec
total size is 45305958  speedup is 2.87

6. Remote shell for Synchronization

rsync allows you to specify the remote shell which you want to use. You can use rsync ssh to enable the secured remote connection. Use rsync -e ssh to specify which remote shell to use.

$ rsync -avz -e ssh root@192.168.200.10:/var/lib/rpm /root/temp
Password:
receiving file list ... done
rpm/
rpm/Basenames
 
sent 406 bytes  received 15810230 bytes  2432405.54 bytes/sec
total size is 45305958  speedup is 2.87

7. Do Not Overwrite the Modified Files at the Destination

In a typical sync situation, if a file is modified at the destination, we might not want to overwrite the file with the old file from the source. Use rsync -u option to do exactly that.

$ ls -l /root/temp/Basenames
total 39088
-rwxr-xr-x 1 root root        4096 Sep  2 11:35 Basenames
 
$ rsync -avzu root@192.168.200.10:/var/lib/rpm /root/temp
Password:
receiving file list ... done
rpm/
 
sent 122 bytes  received 505 bytes  114.00 bytes/sec
total size is 45305958  speedup is 72258.31
 
$ ls -lrt
total 39088
-rwxr-xr-x 1 root root        4096 Sep  2 11:35 Basenames

8. Synchronize only the Directory Tree Structure (not the files)

$ rsync -v -d root@192.168.200.10:/var/lib/ .
Password:
receiving file list ... done
logrotate.status
CAM/
YaST2/
acpi/
 
sent 240 bytes  received 1830 bytes  318.46 bytes/sec
total size is 956  speedup is 0.46

9. View the rsync Progress during Transfer

rsync –progress option displays detailed progress of rsync execution as shown below.

$ rsync -avz --progress root@192.168.200.10:/var/lib/rpm/ /root/temp/
Password:
receiving file list ...
19 files to consider
./
Basenames
     5357568 100%   14.98MB/s    0:00:00 (xfer#1, to-check=17/19)
Conflictname
       12288 100%   35.09kB/s    0:00:00 (xfer#2, to-check=16/19)
.
.
.
sent 406 bytes  received 15810211 bytes  2108082.27 bytes/sec
total size is 45305958  speedup is 2.87

10. Delete the Files Created at the Target

If a file is not present at the source, but present at the target, you might want to delete the file at the target during rsync. In that case, use –delete option as shown below. rsync delete option deletes files that are not there in source directory.

# Source and target are in sync. Now creating new file at the target.
$ > new-file.txt
 
$ rsync -avz --delete root@192.168.200.10:/var/lib/rpm/ .
Password:
receiving file list ... done
deleting new-file.txt
./
 
sent 26 bytes  received 390 bytes  48.94 bytes/sec
total size is 45305958  speedup is 108908.55

Target has the new file called new-file.txt, when synchronize with the source with –delete option, it removed the file new-file.txt

11. Do not Create New File at the Target

If you want this feature, use –existing option with rsync command.

First, add a new-file.txt at the source.

$ > new-file.txt

Next, execute the rsync from the target.

$ rsync -avz --existing root@192.168.1.2:/var/lib/rpm/ .
root@192.168.1.2's password:
receiving file list ... done
./
 
sent 26 bytes  received 419 bytes  46.84 bytes/sec
total size is 88551424  speedup is 198991.96

If you see the above output, it didn’t receive the new file new-file.txt

12. View the Changes Between Source and Destination

This option is useful to view the difference in the files or directories between source and destination.

At the source:

$ ls -l /var/lib/rpm
-rw-r--r-- 1 root root  5357568 2010-06-24 08:57 Basenames
-rw-r--r-- 1 root root    12288 2008-05-28 22:03 Conflictname
-rw-r--r-- 1 root root  1179648 2010-06-24 08:57 Dirnames

At the destination:

$ ls -l /root/temp
-rw-r--r-- 1 root root    12288 May 28  2008 Conflictname
-rw-r--r-- 1 bin  bin   1179648 Jun 24 05:27 Dirnames
-rw-r--r-- 1 root root        0 Sep  3 06:39 Basenames

In the above example, between the source and destination, there are two differences. First, owner and group of the file Dirname differs. Next, size differs for the file Basenames.

Now let us see how rsync displays this difference. -i option displays the item changes.

$ rsync -avzi root@192.168.200.10:/var/lib/rpm/ /root/temp/
Password:
receiving file list ... done
>f.st.... Basenames
.f....og. Dirnames
 
sent 48 bytes  received 2182544 bytes  291012.27 bytes/sec
total size is 45305958  speedup is 20.76

In the output it displays some 9 letters in front of the file name or directory name indicating the changes. In our example, the letters in front of the Basenames (and Dirnames) says the following:

> specifies that a file is being transferred to the local host.
f represents that it is a file.
s represents size changes are there.
t represents timestamp changes are there.
o owner changed
g group changed.

13. Include and Exclude Pattern during File Transfer

rsync allows you to give the pattern you want to include and exclude files or directories while doing synchronization.

$ rsync -avz --include 'P*' --exclude '*' root@192.168.200.10:/var/lib/rpm/ /root/temp/
Password:
receiving file list ... done
./
Packages
Providename
Provideversion
Pubkeys
 
sent 129 bytes  received 10286798 bytes  2285983.78 bytes/sec
total size is 32768000  speedup is 3.19

In the above example, it includes only the files or directories starting with ‘P’ (using rsync include) and excludes all other files. (using rsync exclude ‘*’ )

14. Do Not Transfer Large Files

You can tell rsync not to transfer files that are greater than a specific size using rsync –max-size option.

$ rsync -avz --max-size='100K' root@192.168.200.10:/var/lib/rpm/ /root/temp/
Password:
receiving file list ... done
./
Conflictname
Group
Installtid
Name
Sha1header
Sigmd5
Triggername
 
sent 252 bytes  received 123081 bytes  18974.31 bytes/sec
total size is 45305958  speedup is 367.35

max-size=100K makes rsync to transfer only the files that are less than or equal to 100K. You can indicate M for megabytes and G for gigabytes.

15. Transfer the Whole File

One of the main feature of rsync is that it transfers only the changed block to the destination, instead of sending the whole file.

If network bandwidth is not an issue for you (but CPU is), you can transfer the whole file, using rsync -W option. This will speed-up the rsync process, as it doesn’t have to perform the checksum at the source and destination.

#  rsync -avzW  [email protected]:/var/lib/rpm/ /root/temp
Password:
receiving file list ... done
./
Basenames
Conflictname
Dirnames
Filemd5s
Group
Installtid
Name
 
sent 406 bytes  received 15810211 bytes  2874657.64 bytes/sec
total size is 45305958  speedup is 2.87

15. Do a Dry Run with rsync

If you are a newbie and using rsync and don’t know what exactly your command going do. Rsync could really mess up the things in your destination folder and then doing an undo can be a tedious job.
Use of this option will not make any changes only do a dry run of the command and shows the output of the command, if the output shows exactly same you want to do then you can remove ‘–dry-run‘ option from your command and run on the terminal.

$ rsync --dry-run --remove-source-files -zvh backup.tar /tmp/backups/
 
backup.tar
 
sent 35 bytes  received 15 bytes  100.00 bytes/sec
 
total size is 16.18M  speedup is 323584.00 (DRY RUN)

16. Set Bandwidth Limit and Transfer File

You can set the bandwidth limit while transferring data from one machine to another machine with the the help of ‘–bwlimit‘ option. This options helps us to limit I/O bandwidth.

$ rsync --bwlimit=100 -avzhe ssh  /var/lib/rpm/  root@192.168.0.100:/root/tmprpm/
root@192.168.0.100's password:
sending incremental file list
sent 324 bytes  received 12 bytes  61.09 bytes/sec
total size is 38.08M  speedup is 113347.05

Also, by default rsync syncs changed blocks and bytes only, if you want explicitly want to sync whole file then you use ‘-W‘ option with it.

$ rsync -zvhW backup.tar /tmp/backups/backup.tar
backup.tar
sent 14.71M bytes  received 31 bytes  3.27M bytes/sec
total size is 16.18M  speedup is 1.10