| rsnapshot(1) | rsnapshot-tools | rsnapshot(1) |
rsnapshot - remote filesystem snapshot utility
rsnapshot [-vtxqVD] [-c cfgfile] [command] [args]
rsnapshot is a filesystem snapshot utility. It can take incremental snapshots of local and remote filesystems for any number of machines.
Local filesystem snapshots are handled with rsync(1). Secure remote connections are handled with rsync over ssh(1), while anonymous rsync connections simply use an rsync server. Both remote and local transfers depend on rsync.
rsnapshot saves much more disk space than you might imagine. The amount of space required is roughly the size of one full backup, plus a copy of each additional file that is changed. rsnapshot makes extensive use of hard links, so if the file doesn't change, the next snapshot is simply a hard link to the exact same file.
rsnapshot will typically be invoked as root by a cron job, or series of cron jobs. It is possible, however, to run as any arbitrary user with an alternate configuration file.
All important options are specified in a configuration file, which is located by default at /etc/rsnapshot.conf. An alternate file can be specified on the command line. There are also additional options which can be passed on the command line.
The command line options are as follows:
-t test, show shell commands that would be executed
-c path to alternate config file
-x one filesystem, don't cross partitions within each backup point
-q quiet, suppress non-fatal warnings
-V same as -v, but with more detail
-D a firehose of diagnostic information
/etc/rsnapshot.conf is the default configuration file. All parameters in this file must be separated by tabs. /usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz can be used as a reference.
It is recommended that you copy /usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz to /etc/rsnapshot.conf, and then modify /etc/rsnapshot.conf to suit your needs.
Long lines may be split over several lines. "Continuation" lines must begin with a space or a tab character. Continuation lines will have all leading and trailing whitespace stripped off, and then be appended with an intervening tab character to the previous line when the configuation file is parsed.
Here is a list of allowed parameters:
snapshot_root Local filesystem path to save all snapshots
include_conf Include another file in the configuration at this point.
no_create_root If set to 1, rsnapshot won't create snapshot_root directory
cmd_rsync Full path to rsync (required)
cmd_ssh Full path to ssh (optional)
cmd_cp Full path to cp (optional, but must be GNU version)
With GNU cp, rsnapshot can take care of both normal files and special files (such as FIFOs, sockets, and block/character devices) in one pass.
If cmd_cp is disabled, rsnapshot will use its own built-in function, native_cp_al() to backup up regular files and directories. This will then be followed up by a separate call to rsync, to move the special files over (assuming there are any).
cmd_rm Full path to rm (optional)
cmd_logger Full path to logger (optional, for syslog support)
cmd_du Full path to du (optional, for disk usage reports)
cmd_rsnapshot_diff Full path to rsnapshot-diff (optional)
cmd_preexec
cmd_postexec
linux_lvm_cmd_lvcreate
linux_lvm_cmd_lvremove
linux_lvm_cmd_mount
linux_lvm_cmd_umount
retain [name] [number]
A deprecated alias for 'retain' is 'interval'.
Example: retain alpha 6
[root@localhost]# rsnapshot alpha
For this example, every time this is run, the following will happen:
<snapshot_root>/alpha.5/ will be deleted, if it exists.
<snapshot_root>/alpha.{1,2,3,4} will all be rotated +1, if they exist.
<snapshot_root>/alpha.0/ will be copied to <snapshot_root>/alpha.1/ using hard links.
Each backup point (explained below) will then be rsynced to the corresponding directories in <snapshot_root>/alpha.0/
Backup levels must be specified in the config file in order, from most frequent to least frequent. The first entry is the one which will be synced with the backup points. The subsequent backup levels (e.g., beta, gamma, etc) simply rotate, with each higher backup level pulling from the one below it for its .0 directory.
Example:
retain beta 7
retain gamma 4
beta.0/ will be moved from alpha.5/, and gamma.0/ will be moved from beta.6/
alpha.0/ will be rsynced directly from the filesystem.
link_dest 1
sync_first 1
This allows better recovery in the event that rsnapshot is interrupted in the middle of a sync operation, since the sync step and rotation steps are separated. This also means that you can easily run "rsnapshot sync" on the command line without fear of forcing all the other directories to rotate up. This benefit comes at the cost of one more snapshot worth of disk space. The default is 0 (off).
verbose 2
1 Quiet Show fatal errors only
2 Default Show warnings and errors
3 Verbose Show equivalent shell commands being executed
4 Extra Verbose Same as verbose, but with more detail
5 Debug All kinds of information
loglevel 3
logfile /var/log/rsnapshot
include [file-name-pattern]
exclude [file-name-pattern]
include_file /path/to/include/file
exclude_file /path/to/exclude/file
rsync_short_args -a
"-a" is rsync's "archive mode" which tells it to copy as much of the filesystem metadata as it can for each file. This specifically does *not* include information about hard links, as that would greatly increase rsync's memory usage and slow it down. If you need to preserve hard links in your backups, then add "H" to this.
rsync_long_args --delete --numeric-ids --relative --delete-excluded
Quotes are permitted in rsync_long_args, eg --rsync-path="sudo /usr/bin/rsync". You may use either single (') or double (") quotes, but nested quotes (including mixed nested quotes) are not permitted. Similar quoting is also allowed in per-backup-point rsync_long_args.
ssh_args -p 22
du_args -csh
lockfile /var/run/rsnapshot.pid
stop_on_stale_lockfile 0
If a lockfile exists when rsnapshot starts, it will try to read the file and stop with an error if it can't. If it *can* read the file, it sees if a process exists with the PID noted in the file. If it does, rsnapshot stops with an error message. If there is no process with that PID, then we assume that the lockfile is stale and ignore it *unless* stop_on_stale_lockfile is set to 1 in which case we stop.
stop_on_stale_lockfile defaults to 0.
one_fs 1
use_lazy_deletes 1
Enabling this means that snapshots get taken sooner (since the delete doesn't come first), and any other rsnapshot processes are allowed to start while the final delete is happening. This benefit comes at the cost of using more disk space. The default is 0 (off).
The details of how this works have changed in rsnapshot version 1.3.1. Originally you could only ever have one .delete directory per backup level. Now you can have many, so if your next (eg) alpha backup kicks off while the previous one is still doing a lazy delete you may temporarily have extra _delete directories hanging around.
linux_lvm_snapshotsize 2G
linux_lvm_snapshotname rsnapshot
linux_lvm_vgpath /dev
linux_lvm_mountpath /mnt/lvm-snapshot
backup /etc/ localhost/
backup root@example.com:/etc/ example.com/
backup rsync://example.com/path2/ example.com/
backup /var/ localhost/ one_fs=1
backup lvm://vg0/home/path2/ lvm-vg0/
backup_script /usr/local/bin/backup_pgsql.sh pgsql_backup/
backup /etc/ localhost/
backup /usr/local/ localhost/
backup root@example.com:/etc/ example.com/
backup example.com:/etc/ example.com/
backup root@example.com:/usr/local/ example.com/
backup rsync://example.com/pub/ example.com/pub/
backup /var/ localhost/ one_fs=1
It is most useful when specifying per-backup rsync excludes thus:
backup root@somehost:/ somehost +rsync_long_args=--exclude=/var/spool/
Note the + sign. That tells rsnapshot to add to the list of arguments to pass to rsync instead of replacing the list.
backup lvm://vg0/home/path2/ lvm-vg0/
backup_script /usr/local/bin/backup_database.sh db_backup/
Please note that whatever is in the destination directory will be completely deleted and recreated. For this reason, rsnapshot prevents you from specifying a destination directory for a backup_script that will clobber other backups.
So in this example, say the backup_database.sh script simply runs a command like:
mysqldump -uusername mydatabase > mydatabase.sql
chmod u=r,go= mydatabase.sql # r-------- (0400)
rsnapshot will take the generated "mydatabase.sql" file and move it into the <snapshot_root>/<retain>.0/db_backup/ directory. On subsequent runs, rsnapshot checks the differences between the files created against the previous files. If the backup script generates the same output on the next run, the files will be hard linked against the previous ones, and no additional disk space will be taken up.
backup_exec ssh root@1.2.3.4 "du -sh /.offsite_backup" optional/ backup_exec rsync -az /.snapshots/daily.0 root@1.2.3.4:/.offsite_backup/ required/ backup_exec /bin/true/
Remember that tabs must separate all elements, and that there must be a trailing slash on the end of every directory.
A hash mark (#) on the beginning of a line is treated as a comment.
Putting it all together (an example file):
# THIS IS A COMMENT, REMEMBER TABS MUST SEPARATE ALL ELEMENTS
config_version 1.2
snapshot_root /.snapshots/
cmd_rsync /usr/bin/rsync
cmd_ssh /usr/bin/ssh
#cmd_cp /bin/cp
cmd_rm /bin/rm
cmd_logger /usr/bin/logger
cmd_du /usr/bin/du
linux_lvm_cmd_lvcreate /sbin/lvcreate
linux_lvm_cmd_lvremove /sbin/lvremove
linux_lvm_cmd_mount /bin/mount
linux_lvm_cmd_umount /bin/umount
linux_lvm_snapshotsize 2G
linux_lvm_snapshotname rsnapshot
linux_lvm_vgpath /dev
linux_lvm_mountpath /mnt/lvm-snapshot
retain alpha 6
retain beta 7
retain gamma 7
retain delta 3
backup /etc/ localhost/
backup /home/ localhost/
backup_script /usr/local/bin/backup_mysql.sh mysql_backup/
backup root@foo.com:/etc/ foo.com/
backup root@foo.com:/home/ foo.com/
backup root@mail.foo.com:/home/ mail.foo.com/
backup rsync://example.com/pub/ example.com/pub/
backup lvm://vg0/xen-home/ lvm-vg0/xen-home/
backup_exec echo "backup finished!"
rsnapshot can be used by any user, but for system-wide backups you will probably want to run it as root.
Since backups usually get neglected if human intervention is required, the preferred way is to run it from cron.
When you are first setting up your backups, you will probably also want to run it from the command line once or twice to get a feel for what it's doing.
Here is an example crontab entry, assuming that backup levels alpha, beta, gamma and delta have been defined in /etc/rsnapshot.conf
50 23 * * * /usr/bin/rsnapshot beta
40 23 * * 6 /usr/bin/rsnapshot gamma
30 23 1 * * /usr/bin/rsnapshot delta
This example will do the following:
1 beta backup every day, at 11:50PM
1 gamma backup every week, at 11:40PM, on Saturdays (6th day of week)
1 delta backup every month, at 11:30PM on the 1st day of the month
It is usually a good idea to schedule the larger backup levels to run a bit before the lower ones. For example, in the crontab above, notice that "beta" runs 10 minutes before "alpha". The main reason for this is that the beta rotate will pull out the oldest alpha and make that the youngest beta (which means that the next alpha rotate will not need to delete the oldest alpha), which is more efficient. A secondary reason is that it is harder to predict how long the lowest backup level will take, since it needs to actually do an rsync of the source as well as the rotate that all backups do.
If rsnapshot takes longer than 10 minutes to do the "beta" rotate (which usually includes deleting the oldest beta snapshot), then you should increase the time between the backup levels. Otherwise (assuming you have set the lockfile parameter, as is recommended) your alpha snapshot will fail sometimes because the beta still has the lock.
Remember that these are just the times that the program runs. To set the number of backups stored, set the retain numbers in /etc/rsnapshot.conf
To check the disk space used by rsnapshot, you can call it with the "du" argument.
For example:
This will show you exactly how much disk space is taken up in the snapshot root. This feature requires the UNIX du command to be installed on your system, for it to support the "-csh" command line arguments, and to be in your path. You can also override your path settings and the flags passed to du using the cmd_du and du_args parameters.
It is also possible to pass a relative file path as a second argument, to get a report on a particular file or subdirectory.
The GNU version of "du" is preferred. The BSD version works well also, but does not support the -h flag (use -k instead, to see the totals in kilobytes). Other versions of "du", such as Solaris, may not work at all.
To check the differences between two directories, call rsnapshot with the "diff" argument, followed by two backup levels or directory paths.
For example:
rsnapshot diff beta.0/localhost/etc beta.1/localhost/etc
rsnapshot diff /.snapshots/beta.0 /.snapshots/beta.1
This will call the rsnapshot-diff program, which will scan both directories looking for differences (based on hard links).
rsnapshot sync
50 23 * * * /usr/bin/rsnapshot beta
40 23 1,8,15,22 * * /usr/bin/rsnapshot gamma
30 23 1 * * /usr/bin/rsnapshot delta
The sync operation simply runs rsync and all backup scripts. In this scenario, all calls simply rotate directories, even the lowest backup level.
rsnapshot sync [dest]
For example, let's say that example.com is a destination path shared by one or more of your backup points.
This command will only sync the files that normally get backed up into example.com. It will NOT get any other backup points with slightly different values (like example.com/etc/, for example). In order to sync example.com/etc, you would need to run rsnapshot again, using example.com/etc as the optional parameter.
rsnapshot configtest
1 A fatal error occurred
2 Some warnings occurred, but the backup still finished
/etc/rsnapshot.conf
rsync(1), ssh(1), logger(1), sshd(1), ssh-keygen(1), perl(1), cp(1), du(1), crontab(1)
Use the -t flag to see what commands would have been executed. This will show you the commands rsnapshot would try to run. There are a few minor differences (for example, not showing an attempt to remove the lockfile because it wasn't really created in the test), but should give you a very good idea what will happen.
Using the -v, -V, and -D flags will print increasingly more information to STDOUT.
Make sure you don't have spaces in the config file that you think are actually tabs.
Much other weird behavior can probably be attributed to plain old file system permissions and ssh authentication issues.
Please report bugs (and other comments) to the rsnapshot-discuss mailing list:
http://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Make sure your /etc/rsnapshot.conf file has all elements separated by tabs. See /usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz for a working example file.
Make sure you put a trailing slash on the end of all directory references. If you don't, you may have extra directories created in your snapshots. For more information on how the trailing slash is handled, see the rsync(1) manpage.
Make sure to make the snapshot directory chmod 700 and owned by root (assuming backups are made by the root user). If the snapshot directory is readable by other users, they will be able to modify the snapshots containing their files, thus destroying the integrity of the snapshots.
If you would like regular users to be able to restore their own backups, there are a number of ways this can be accomplished. One such scenario would be:
Set snapshot_root to /.private/.snapshots in /etc/rsnapshot.conf
Set the file permissions on these directories as follows:
drwxr-xr-x /.private/.snapshots
Export the /.private/.snapshots directory over read-only NFS, a read-only Samba share, etc.
See the rsnapshot HOWTO for more information on making backups accessible to non-privileged users.
For ssh to work unattended through cron, you will probably want to use public key logins. Create an ssh key with no passphrase for root, and install the public key on each machine you want to backup. If you are backing up system files from remote machines, this probably means unattended root logins. Another possibility is to create a second user on the machine just for backups. Give the user a different name such as "rsnapshot", but keep the UID and GID set to 0, to give root privileges. However, make logins more restrictive, either through ssh configuration, or using an alternate shell.
BE CAREFUL! If the private key is obtained by an attacker, they will have free run of all the systems involved. If you are unclear on how to do this, see ssh(1), sshd(1), and ssh-keygen(1).
Backup scripts are run as the same user that rsnapshot is running as. Typically this is root. Make sure that all of your backup scripts are only writable by root, and that they don't call any other programs that aren't owned by root. If you fail to do this, anyone who can write to the backup script or any program it calls can fully take over the machine. Of course, this is not a situation unique to rsnapshot.
By default, rsync transfers are done using the --numeric-ids option. This means that user names and group names are ignored during transfers, but the UID/GID information is kept intact. The assumption is that the backups will be restored in the same environment they came from. Without this option, restoring backups for multiple heterogeneous servers would be unmanageable. If you are archiving snapshots with GNU tar, you may want to use the --numeric-owner parameter. Also, keep a copy of the archived system's /etc/passwd and /etc/group files handy for the UID/GID to name mapping.
If you remove backup points in the config file, the previously archived files under those points will permanently stay in the snapshots directory unless you remove the files yourself. If you want to conserve disk space, you will need to go into the <snapshot_root> directory and manually remove the files from the smallest backup level's ".0" directory.
For example, if you were previously backing up /home/ with a destination of localhost/, and alpha is your smallest backup level, you would need to do the following to reclaim that disk space:
Please note that the other snapshots previously made of /home/ will still be using that disk space, but since the files are flushed out of alpha.0/, they will no longer be copied to the subsequent directories, and will thus be removed in due time as the rotations happen.
Mike Rubel - http://www.mikerubel.org/computers/rsync_snapshots/
Nathan Rosenquist (nathan@rsnapshot.org)
David Cantrell (david@cantrell.org.uk)
David Keegel <djk@cybersource.com.au>
Benedikt Heine <benedikt@heine.rocks>
Carl Wilhelm Soderstrom (chrome@real-time.com)
Ted Zlatanov (tzz@lifelogs.com)
Ralf van Dooren (r.vdooren@snow.nl)
SlapAyoda
Carl Boe (boe@demog.berkeley.edu)
Shane Leibling (shane@cryptio.net)
Christoph Wegscheider (christoph.wegscheider@wegi.net)
Bharat Mediratta (bharat@menalto.com)
Peter Palfrader (weasel@debian.org)
Nicolas Kaiser (nikai@nikai.net)
Chris Petersen - (http://www.forevermore.net/)
Robert Jackson (RobertJ@promedicalinc.com)
Justin Grote (justin@grote.name)
Anthony Ettinger (apwebdesign@yahoo.com)
Sherman Boyd
William Bear (bear@umn.edu)
Eric Anderson (anderson@centtech.com)
Alan Batie (alan@batie.org)
Dieter Bloms (dieter@bloms.de)
Henning Moll (newsScott@gmx.de)
Ben Low (ben@bdlow.net)
Copyright (C) 2003-2005 Nathan Rosenquist
Portions Copyright (C) 2002-2007 Mike Rubel, Carl Wilhelm Soderstrom, Ted Zlatanov, Carl Boe, Shane Liebling, Bharat Mediratta, Peter Palfrader, Nicolas Kaiser, David Cantrell, Chris Petersen, Robert Jackson, Justin Grote, David Keegel, Alan Batie, Dieter Bloms, Henning Moll, Ben Low, Anthony Ettinger
This man page is distributed under the same license as rsnapshot: the GPL (see below).
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
| 2016-03-26 | rsnapshot-tools |