rsnapshot - Online in the Cloud

PROGRAM:

NAME

rsnapshot - remote filesystem snapshot utility

SYNOPSIS

rsnapshot [-vtxqVD] [-c cfgfile] [command] [args]

DESCRIPTION

rsnapshot is a filesystem snapshot utility. It can take incremental snapshots of local and
remote filesystems for any number of machines.

Local filesystem snapshots are handled with rsync(1). Secure remote connections are
handled with rsync over ssh(1), while anonymous rsync connections simply use an rsync
server. Both remote and local transfers depend on rsync.

rsnapshot saves much more disk space than you might imagine. The amount of space required
is roughly the size of one full backup, plus a copy of each additional file that is
changed. rsnapshot makes extensive use of hard links, so if the file doesn't change, the
next snapshot is simply a hard link to the exact same file.

rsnapshot will typically be invoked as root by a cron job, or series of cron jobs. It is
possible, however, to run as any arbitrary user with an alternate configuration file.

All important options are specified in a configuration file, which is located by default
at /etc/rsnapshot.conf. An alternate file can be specified on the command line. There are
also additional options which can be passed on the command line.

The command line options are as follows:

-v verbose, show shell commands being executed

-t test, show shell commands that would be executed

-c path to alternate config file

-x one filesystem, don't cross partitions within each backup point

-q quiet, suppress non-fatal warnings

-V same as -v, but with more detail

-D a firehose of diagnostic information

CONFIGURATION

/etc/rsnapshot.conf is the default configuration file. All parameters in this file must be
separated by tabs. /usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz can be used
as a reference.

It is recommended that you copy
/usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz to /etc/rsnapshot.conf, and
then modify /etc/rsnapshot.conf to suit your needs.

Long lines may be split over several lines. "Continuation" lines must begin with a space
or a tab character. Continuation lines will have all leading and trailing whitespace
stripped off, and then be appended with an intervening tab character to the previous line
when the configuation file is parsed.

Here is a list of allowed parameters:

config_version Config file version (required). Default is 1.2

snapshot_root Local filesystem path to save all snapshots

include_conf Include another file in the configuration at this point.

This is recursive, but you may need to be careful about paths when specifying
which file to include. We check to see if the file you have specified is
readable, and will yell an error if it isn't. We recommend using a full path. As
a special case, include_conf's value may be enclosed in `backticks` in which case
it will be executed and whatever it spits to STDOUT will be included in the
configuration. Note that shell meta-characters may be interpreted.

no_create_root If set to 1, rsnapshot won't create snapshot_root directory

cmd_rsync Full path to rsync (required)

cmd_ssh Full path to ssh (optional)

cmd_cp Full path to cp (optional, but must be GNU version)

If you are using Linux, you should uncomment cmd_cp. If you are using a platform
which does not have GNU cp, you should leave cmd_cp commented out.

With GNU cp, rsnapshot can take care of both normal files and special files (such
as FIFOs, sockets, and block/character devices) in one pass.

If cmd_cp is disabled, rsnapshot will use its own built-in function,
native_cp_al() to backup up regular files and directories. This will then be
followed up by a separate call to rsync, to move the special files over (assuming
there are any).

cmd_rm Full path to rm (optional)

cmd_logger Full path to logger (optional, for syslog support)

cmd_du Full path to du (optional, for disk usage reports)

cmd_rsnapshot_diff Full path to rsnapshot-diff (optional)

cmd_preexec

Full path (plus any arguments) to preexec script (optional). This script will run
immediately before each backup operation (but not any rotations). If the execution
fails, rsnapshot will stop immediately.

cmd_postexec

Full path (plus any arguments) to postexec script (optional). This script will
run immediately after each backup operation (but not any rotations). If the
execution fails, rsnapshot will stop immediately.

linux_lvm_cmd_lvcreate

linux_lvm_cmd_lvremove

linux_lvm_cmd_mount

linux_lvm_cmd_umount

Paths to lvcreate, lvremove, mount and umount commands, for use with Linux LVMs.
You may include options to the commands also. The lvcreate, lvremove, mount and
umount commands are required for managing snapshots of LVM volumes and are
otherwise optional.

retain [name] [number]

"name" refers to the name of this backup level (e.g., alpha, beta, so also called
the 'interval'). "number" is the number of snapshots for this type of interval
that will be retained. The value of "name" will be the command passed to
rsnapshot to perform this type of backup.

A deprecated alias for 'retain' is 'interval'.

Example: retain alpha 6

[root@localhost]# rsnapshot alpha

For this example, every time this is run, the following will happen:

<snapshot_root>/alpha.5/ will be deleted, if it exists.

<snapshot_root>/alpha.{1,2,3,4} will all be rotated +1, if they exist.

<snapshot_root>/alpha.0/ will be copied to <snapshot_root>/alpha.1/ using hard
links.

Each backup point (explained below) will then be rsynced to the corresponding
directories in <snapshot_root>/alpha.0/

Backup levels must be specified in the config file in order, from most frequent to
least frequent. The first entry is the one which will be synced with the backup
points. The subsequent backup levels (e.g., beta, gamma, etc) simply rotate, with
each higher backup level pulling from the one below it for its .0 directory.

Example:

retain alpha 6

retain beta 7

retain gamma 4

beta.0/ will be moved from alpha.5/, and gamma.0/ will be moved from beta.6/

alpha.0/ will be rsynced directly from the filesystem.

link_dest 1

If your version of rsync supports --link-dest (2.5.7 or newer), you can enable
this to let rsync handle some things that GNU cp or the built-in subroutines would
otherwise do. Enabling this makes rsnapshot take a slightly more complicated code
branch, but it's the best way to support special files on non-Linux systems.

sync_first 1

sync_first changes the behaviour of rsnapshot. When this is enabled, all calls to
rsnapshot with various backup levels simply rotate files. All backups are handled
by calling rsnapshot with the "sync" argument. The synced files are stored in a
".sync" directory under the snapshot_root.

This allows better recovery in the event that rsnapshot is interrupted in the
middle of a sync operation, since the sync step and rotation steps are separated.
This also means that you can easily run "rsnapshot sync" on the command line
without fear of forcing all the other directories to rotate up. This benefit
comes at the cost of one more snapshot worth of disk space. The default is 0
(off).

verbose 2

The amount of information to print out when the program is run. Allowed values are
1 through 5. The default is 2.

1 Quiet Show fatal errors only
2 Default Show warnings and errors
3 Verbose Show equivalent shell commands being executed
4 Extra Verbose Same as verbose, but with more detail
5 Debug All kinds of information

loglevel 3

This number means the same thing as verbose above, but it determines how much data
is written to the logfile, if one is being written.

logfile /var/log/rsnapshot

Full filesystem path to the rsnapshot log file. If this is defined, a log file
will be written, with the amount of data being controlled by loglevel. If this is
commented out, no log file will be written.

include [file-name-pattern]

This gets passed directly to rsync using the --include directive. This parameter
can be specified as many times as needed, with one pattern defined per line. See
the rsync(1) man page for the syntax.

exclude [file-name-pattern]

This gets passed directly to rsync using the --exclude directive. This parameter
can be specified as many times as needed, with one pattern defined per line. See
the rsync(1) man page for the syntax.

include_file /path/to/include/file

This gets passed directly to rsync using the --include-from directive. See the
rsync(1) man page for the syntax.

exclude_file /path/to/exclude/file

This gets passed directly to rsync using the --exclude-from directive. See the
rsync(1) man page for the syntax.

rsync_short_args -a

List of short arguments to pass to rsync. If not specified, "-a" is the default.
Please note that these must be all next to each other. For example, "-az" is
valid, while "-a -z" is not.

"-a" is rsync's "archive mode" which tells it to copy as much of the filesystem
metadata as it can for each file. This specifically does *not* include
information about hard links, as that would greatly increase rsync's memory usage
and slow it down. If you need to preserve hard links in your backups, then add
"H" to this.

rsync_long_args --delete --numeric-ids --relative --delete-excluded

List of long arguments to pass to rsync. The default values are
--delete --numeric-ids --relative --delete-excluded This means that the
directory structure in each backup point destination will match that in the backup
point source.

Quotes are permitted in rsync_long_args, eg --rsync-path="sudo /usr/bin/rsync".
You may use either single (') or double (") quotes, but nested quotes (including
mixed nested quotes) are not permitted. Similar quoting is also allowed in per-
backup-point rsync_long_args.

ssh_args -p 22

Arguments to be passed to ssh. If not specified, the default is none.

du_args -csh

Arguments to be passed to du. If not specified, the default is -csh. GNU du
supports -csh, BSD du supports -csk, Solaris du doesn't support -c at all. The GNU
version is recommended, since it offers the most features.

lockfile /var/run/rsnapshot.pid

stop_on_stale_lockfile 0

Lockfile to use when rsnapshot is run. This prevents a second invocation from
clobbering the first one. If not specified, no lock file is used. Make sure to
use a directory that is not world writeable for security reasons. Use of a lock
file is strongly recommended.

If a lockfile exists when rsnapshot starts, it will try to read the file and stop
with an error if it can't. If it *can* read the file, it sees if a process exists
with the PID noted in the file. If it does, rsnapshot stops with an error
message. If there is no process with that PID, then we assume that the lockfile
is stale and ignore it *unless* stop_on_stale_lockfile is set to 1 in which case
we stop.

stop_on_stale_lockfile defaults to 0.

one_fs 1

Prevents rsync from crossing filesystem partitions. Setting this to a value of 1
enables this feature. 0 turns it off. This parameter is optional. The default is
0 (off).

use_lazy_deletes 1

Changes default behavior of rsnapshot and does not initially remove the oldest
snapshot. Instead it moves that directory to _delete.[processid] and continues as
normal. Once the backup has been completed, the lockfile will be removed before
rsnapshot starts deleting the directory.

Enabling this means that snapshots get taken sooner (since the delete doesn't come
first), and any other rsnapshot processes are allowed to start while the final
delete is happening. This benefit comes at the cost of using more disk space. The
default is 0 (off).

The details of how this works have changed in rsnapshot version 1.3.1. Originally
you could only ever have one .delete directory per backup level. Now you can have
many, so if your next (eg) alpha backup kicks off while the previous one is still
doing a lazy delete you may temporarily have extra _delete directories hanging
around.

linux_lvm_snapshotsize 2G

LVM snapshot(s) size (lvcreate --size option).

linux_lvm_snapshotname rsnapshot

Name to be used when creating the LVM logical volume snapshot(s) (lvcreate --name
option).

linux_lvm_vgpath /dev

Path to the LVM Volume Groups.

linux_lvm_mountpath /mnt/lvm-snapshot

Mount point to use to temporarily mount the snapshot(s).

backup /etc/ localhost/

backup [email protected]:/etc/ example.com/

backup rsync://example.com/path2/ example.com/

backup /var/ localhost/ one_fs=1

backup lvm://vg0/home/path2/ lvm-vg0/

backup_script /usr/local/bin/backup_pgsql.sh pgsql_backup/

Examples:

backup /etc/ localhost/

Backs up /etc/ to <snapshot_root>/<retain>.0/localhost/etc/ using rsync on the
local filesystem

backup /usr/local/ localhost/

Backs up /usr/local/ to <snapshot_root>/<retain>.0/localhost/usr/local/ using
rsync on the local filesystem

backup [email protected]:/etc/ example.com/

Backs up [email protected]:/etc/ to <snapshot_root>/<retain>.0/example.com/etc/
using rsync over ssh

backup example.com:/etc/ example.com/

Same thing but let ssh choose the remote username (as specified in
~/.ssh/config, otherwise the same as the local username)

backup [email protected]:/usr/local/ example.com/

Backs up [email protected]:/usr/local/ to
<snapshot_root>/<retain>.0/example.com/usr/local/ using rsync over ssh

backup rsync://example.com/pub/ example.com/pub/

Backs up rsync://example.com/pub/ to
<snapshot_root>/<retain>.0/example.com/pub/ using an anonymous rsync server.
Please note that unlike backing up local paths and using rsync over ssh, rsync
servers have "modules", which are top level directories that are exported.
Therefore, the module should also be specified in the destination path, as
shown in the example above (the pub/ directory at the end).

backup /var/ localhost/ one_fs=1

This is the same as the other examples, but notice the fourth column. This is
how you specify per-backup-point options to over-ride global settings. This
extra parameter can take several options, separated by commas.

It is most useful when specifying per-backup rsync excludes thus:

backup root@somehost:/ somehost +rsync_long_args=--exclude=/var/spool/

Note the + sign. That tells rsnapshot to add to the list of arguments to pass
to rsync instead of replacing the list.

backup lvm://vg0/home/path2/ lvm-vg0/

Backs up the LVM logical volume called home, of volume group vg0, to
<snapshot_root>/<interval>.0/lvm-vg0/. Will create, mount, backup, unmount and
remove an LVM snapshot for each lvm:// entry.

backup_script /usr/local/bin/backup_database.sh db_backup/

In this example, we specify a script or program to run. This script should
simply create files and/or directories in its current working directory.
rsnapshot will then take that output and move it into the directory specified
in the third column.

Please note that whatever is in the destination directory will be completely
deleted and recreated. For this reason, rsnapshot prevents you from specifying
a destination directory for a backup_script that will clobber other backups.

So in this example, say the backup_database.sh script simply runs a command
like:

#!/bin/sh

mysqldump -uusername mydatabase > mydatabase.sql

chmod u=r,go= mydatabase.sql # r-------- (0400)

rsnapshot will take the generated "mydatabase.sql" file and move it into the
<snapshot_root>/<retain>.0/db_backup/ directory. On subsequent runs, rsnapshot
checks the differences between the files created against the previous files.
If the backup script generates the same output on the next run, the files will
be hard linked against the previous ones, and no additional disk space will be
taken up.

backup_exec ssh [email protected] "du -sh /.offsite_backup"
optional/ backup_exec rsync -az /.snapshots/daily.0
[email protected]:/.offsite_backup/ required/ backup_exec /bin/true/

backup_exec simply runs the command listed. The second argument is not
required and defaults to a value of 'optional'. It specifies the importance
that the command return 0. Valid values are 'optional' and 'required'. If the
command is specified as optional, a non-zero exit status from the command will
result in a warning message being output. If the command is specified as
'required', a non-zero exit status from the command will result in an error
message being output and rsnapshot itself will exit with a non-zero exit
status.

Remember that tabs must separate all elements, and that there must be a trailing slash
on the end of every directory.

A hash mark (#) on the beginning of a line is treated as a comment.

Putting it all together (an example file):

# THIS IS A COMMENT, REMEMBER TABS MUST SEPARATE ALL ELEMENTS

config_version 1.2

snapshot_root /.snapshots/

cmd_rsync /usr/bin/rsync
cmd_ssh /usr/bin/ssh
#cmd_cp /bin/cp
cmd_rm /bin/rm
cmd_logger /usr/bin/logger
cmd_du /usr/bin/du

linux_lvm_cmd_lvcreate /sbin/lvcreate
linux_lvm_cmd_lvremove /sbin/lvremove
linux_lvm_cmd_mount /bin/mount
linux_lvm_cmd_umount /bin/umount

linux_lvm_snapshotsize 2G
linux_lvm_snapshotname rsnapshot
linux_lvm_vgpath /dev
linux_lvm_mountpath /mnt/lvm-snapshot

retain alpha 6
retain beta 7
retain gamma 7
retain delta 3

backup /etc/ localhost/
backup /home/ localhost/
backup_script /usr/local/bin/backup_mysql.sh mysql_backup/

backup [email protected]:/etc/ foo.com/
backup [email protected]:/home/ foo.com/
backup [email protected]:/home/ mail.foo.com/
backup rsync://example.com/pub/ example.com/pub/
backup lvm://vg0/xen-home/ lvm-vg0/xen-home/
backup_exec echo "backup finished!"

USAGE

rsnapshot can be used by any user, but for system-wide backups you will probably want to
run it as root.

Since backups usually get neglected if human intervention is required, the preferred way
is to run it from cron.

When you are first setting up your backups, you will probably also want to run it from the
command line once or twice to get a feel for what it's doing.

Here is an example crontab entry, assuming that backup levels alpha, beta, gamma and delta
have been defined in /etc/rsnapshot.conf

0 */4 * * * /usr/bin/rsnapshot alpha

50 23 * * * /usr/bin/rsnapshot beta

40 23 * * 6 /usr/bin/rsnapshot gamma

30 23 1 * * /usr/bin/rsnapshot delta

This example will do the following:

6 alpha backups a day (once every 4 hours, at 0,4,8,12,16,20)

1 beta backup every day, at 11:50PM

1 gamma backup every week, at 11:40PM, on Saturdays (6th day of week)

1 delta backup every month, at 11:30PM on the 1st day of the month

It is usually a good idea to schedule the larger backup levels to run a bit before the
lower ones. For example, in the crontab above, notice that "beta" runs 10 minutes before
"alpha". The main reason for this is that the beta rotate will pull out the oldest alpha
and make that the youngest beta (which means that the next alpha rotate will not need to
delete the oldest alpha), which is more efficient. A secondary reason is that it is
harder to predict how long the lowest backup level will take, since it needs to actually
do an rsync of the source as well as the rotate that all backups do.

If rsnapshot takes longer than 10 minutes to do the "beta" rotate (which usually includes
deleting the oldest beta snapshot), then you should increase the time between the backup
levels. Otherwise (assuming you have set the lockfile parameter, as is recommended) your
alpha snapshot will fail sometimes because the beta still has the lock.

Remember that these are just the times that the program runs. To set the number of
backups stored, set the retain numbers in /etc/rsnapshot.conf

To check the disk space used by rsnapshot, you can call it with the "du" argument.

For example:

rsnapshot du

This will show you exactly how much disk space is taken up in the snapshot root. This
feature requires the UNIX du command to be installed on your system, for it to support the
"-csh" command line arguments, and to be in your path. You can also override your path
settings and the flags passed to du using the cmd_du and du_args parameters.

It is also possible to pass a relative file path as a second argument, to get a report on
a particular file or subdirectory.

rsnapshot du localhost/home/

The GNU version of "du" is preferred. The BSD version works well also, but does not
support the -h flag (use -k instead, to see the totals in kilobytes). Other versions of
"du", such as Solaris, may not work at all.

To check the differences between two directories, call rsnapshot with the "diff" argument,
followed by two backup levels or directory paths.

For example:

rsnapshot diff beta.0 beta.1

rsnapshot diff beta.0/localhost/etc beta.1/localhost/etc

rsnapshot diff /.snapshots/beta.0 /.snapshots/beta.1

This will call the rsnapshot-diff program, which will scan both directories looking for
differences (based on hard links).

rsnapshot sync

When sync_first is enabled, rsnapshot must first be called with the sync argument,
followed by the other usual cron entries. The sync should happen as the lowest, most
frequent backup level, and right before. For example:

0 */4 * * * /usr/bin/rsnapshot sync && /usr/bin/rsnapshot alpha

50 23 * * * /usr/bin/rsnapshot beta

40 23 1,8,15,22 * * /usr/bin/rsnapshot gamma

30 23 1 * * /usr/bin/rsnapshot delta

The sync operation simply runs rsync and all backup scripts. In this scenario, all
calls simply rotate directories, even the lowest backup level.

rsnapshot sync [dest]

When sync_first is enabled, all sync behaviour happens during an additional sync step
(see above). When using the sync argument, it is also possible to specify a backup
point destination as an optional parameter. If this is done, only backup points
sharing that destination path will be synced.

For example, let's say that example.com is a destination path shared by one or more of
your backup points.

rsnapshot sync example.com

This command will only sync the files that normally get backed up into example.com.
It will NOT get any other backup points with slightly different values (like
example.com/etc/, for example). In order to sync example.com/etc, you would need to
run rsnapshot again, using example.com/etc as the optional parameter.

rsnapshot configtest

Do a quick sanity check to make sure everything is ready to go.

EXIT VALUES

0 All operations completed successfully

1 A fatal error occurred

2 Some warnings occurred, but the backup still finished

Use rsnapshot online using onworks.net services