syrep - Online in the Cloud

PROGRAM:

NAME

syrep - A file repository synchronization tool

SYNOPSIS

syrep [options...] --list SNAPSHOT ...

syrep [options...] --info SNAPSHOT ...

syrep [options...] --history SNAPSHOT ...

syrep [options...] --dump SNAPSHOT ...

syrep [options...] --update DIRECTORY ...

syrep [options...] --diff SNAPSHOT SNAPSHOT

syrep [options...] --merge SNAPSHOT DIRECTORY

syrep [options...] --merge PATCH DIRECTORY

syrep [options...] --merge DIRECTORY DIRECTORY

syrep [options...] --makepatch DIRECTORY SNAPSHOT

syrep [options...] --extract SNAPSHOT ...

syrep [options...] --cleanup DIRECTORY ...

syrep [options...] --forget SNAPSHOT ...

DESCRIPTION

Syrep is a generic file repository synchronization tool. It may be used to synchronize
large file hierarchies bidirectionally by exchanging patch files. Syrep is truely peer-to-
peer, no central servers are involved. Synchronizations between more than two repositories
are supported. The patch files may be transferred via offline media, e.g. removable hard
disks or compact discs.

Files are tracked by their message digests, currently MD5.

Syrep was written to facilitate the synchronization of two large digital music
repositories without direct network connection. Patch files of several gigabytes are
common in this situation.

Syrep is able to cope with 64 bit file sizes. (LFS)

Syrep is optimized for speed. It may make use of a message digest cache to accelerate the
calculation of digests of a whole directory hierarchy.

A syrep repository is a normal UNIX directory tree containing a special directory .syrep
with a file current.syrep (called snapshot) which holds file system history data. A
directory is turned into a syrep repository by running --update on it. Snapshots are used
to perform basic tasks like comparing repositories (command --diff) or creating patches
between them (command --make-patch).

Syrep will ignore all files and directories that have the extended attribute user.syrep
set to omit. Currently, this works on Linux only. Keep in mind, that only newer kernel
versions and some file system support extended attributes. See attr(5) for more
information.

GENERAL OPTIONS

Exactly one command has to be specified on the command line. On the other hand multiple
options are allowed.

-v | --verbose (option)
Enable more verbose operation

-T | --local-temp (option)
Use temporary directory inside repository. This is very useful when the file
repository you apply patches to is on a different partition than /tmp, because
files my be hard linked instead of copied. This requires a read-writable file
system however.

--ignore-origin (option)
Normally syrep warns you if you update, merge or makepatch a repository with a
matching snapshot not generated on the original host, and asks the user if he
really wants to proceed. This option may be used to disable this question.

-z | --compress (option)
Compress output snapshots or patches. This may slow down syrep extraordinarily and
is more or less useless if the data to compress is already compressed. I suggest
using it for --update but not for --makepatch if the file repository contains MP3
or MPEG files only.

-p | --progress (option)
Show a rotating dash while executing operations

GENERAL COMMANDS

-h | --help (command)
Print help and exit

-V | --version (command)
Print version information and exit

LISTING SNAPSHOT CONTENTS

--list (command)
Command for listing the file log of a repository snapshot

--show-deleted (option)
Show deleted entries

--show-by-md (option)
Show files by message digests. This option collides with --sort.

--show-times (option)
Show first and last seen times

--sort (option)
Sort file listing chronologically. This option collides with --show-by-md.

SHOWING SNAPSHOT INFORMATION

--info (command)
Show information about a repository or snapshot, such as origin, current timestamp
and version, database size.

SHOWING SNAPSHOT HISTORY

--history (command)
Show the version and timestamp history of a snapshot's updates

DUMPING SNAPSHOT CONTENTS

--dump (command)
Show a structure dump of a repository or snapshot

UPDATING A SNAPSHOT

--update (command)
Update or create a snapshot for a repository directory. That is: iterate through
the specified hierarchy and update the snapshot log information accordingly.

-SSTRING | --snapshot=STRING (option)
Use the specified snapshot file instead of the one contained in the repository
directory. This may be helpful if your file hierarchy is read only.

-CSTRING | --cache=STRING (option)
Use the specified message digest cache file instead of the one contained in the
repository directory. This may be helpful if your file hierarchy is read only or
when you plan to maintain a system wide message digest cache. In the latter case
you should use --no-purge as well.

--no-cache (option)
Don't use a message digest cache.

--no-purge (option)
Don't purge obsolete entries from cache after update run. The may be useful if you
plan to maintain a system wide message digest cache.

--ro-cache (option)
Use cache in a read only fashion

--check-dev (option)
Store information about the device where the file resides when storing an entry
about it in the message digest cache. Since nowadays device identifiers cannot be
longer considered stable, this options defaults to off.

SHOWING DIFFERENCES BETWEEN TWO SNAPSHOTS

--diff (command)
Show difference between two repositories or snapshots

-s | --sizes (option)
Show the sizes of the files to copy. This works only when acting on a local
repository and for the local files.

-H | --human-readable (option)
Only useful when using with -s. When enabled shows file sizes in human readable
form, i.e. "3.5 MB" instead of "3670016".

MERGING A SNAPSHOT, A PATCH OR A REPOSITORY INTO A REPOSITORY

--merge (command)
Merge a snapshot, a patch or a repository into a repository. Afterwards, you should
run --update on the repository to update the snapshot.

-q | --question (option)
Ask a question before each action

-P | --prune-empty (option)
Prune empty directories

--keep-trash (option)
Don't empty trash. Deleted files are copied into a trash folder inside the
repository directory. If this option is specified this trash is not emptied when
the operation is completed.

--check-md (option)
Check message digests of files before deleting or replacing them. NB: This worsens
syrep's performance and is thus not enabled by default.

--always-copy (option)
As default syrep tries to hard link files instead of copying them. With this option
syrep will always do a copy, regardless if a hard link is possible or not. This is
especially useful when doing bi-directory merges. NB: This worsens syrep's
performance and is thus not enabled by default.

MAKING A PATCH FOR A SNAPSHOT AGAINST A DIRECTORY

--makepatch (command)
Make a patch against the specified repository. The patch is written to STDOUT
unless -o is specified.

-oSTRING | --output-file=STRING (option)
Write output to specified file instead of STDOUT

--include-all (option)
Include files in patch which do exist on the other side under a different name

EXTRACTING A SNAPSHOT'S CONTENTS

--extract (command)
Extract the contents of a snapshot or patch to the local directory unless -D is
specified.

-DSTRING | --output-directory=STRING (option)
Write output to specified directory

CLEANING UP A REPOSITORY

--cleanup (command)
Remove syrep info from repository

-lINT | --cleanup-level=INT
1: just remove temporary data and trash (default); 2: remove message digest cache
as well; 3: remove all syrep data

FORGETTING OUTDATED SNAPSHOT INFORMATION

--forget (command)
Repackage the snapshot file dropping outdated information. Use this if your
snapshot files get too large.

-R DAYS | --remember=DAYS (option)
Information about how many days in the past should be kept? This defaults to 180
(half a year).

REPOSITORIES, SNAPSHOTS AND PATCHES

A syrep file repository is a POSIX file hierarchy with some additional log data, which is
used to track changes. Normally this log data is saved as "snapshot" in the file
$(REPOSITORY)/.syrep/current.syrep. You may create and update it by running --update. The
more often this log is updated the better modifications may be tracked. Therefore this
operation should be called at least once a day via cron(8)

Two snapshots of two distinct repositories (possibly from different hosts) may be compared
with ---diff. This will show you which files should be copied or deleted from or to the
other repository. --makepatch will attach the data of the local missing in the remote
repository to a snapshot and write it to a patch file. This file should be transferred to
the other repository and applied there with --merge.

Keep in mind that patches contain the snapshot data of the originating host. Because of
that you may use it as a snapshot, e.g. by running --diff on it. On the other hand you are
also able to merge snapshots without attached patch data to a repository. This will do all
required deletions and renames, but naturally won't add any new data to the file tree.

To extract the contents of a patch you may use --extract. This will write all files
contained in the patch or snapshot to the local directory, including snapshot log data.
Files are named by their message digests.

Use syrep online using onworks.net services