cback3-amazons3-sync - Synchronize a local directory with an
Amazon S3 bucket
cback3-amazons3-sync [switches] sourceDir
s3BucketUrl
This is the Cedar Backup 3 Amazon S3 sync tool. It synchronizes a
local directory to an Amazon S3 cloud storage bucket. After the sync is
complete, a validation step is taken. An error is reported if the contents
of the bucket do not match the source directory, or if the indicated size
for any file differs.
Generally, one can run the cback3-amazons3-sync command with no
special switches. This will start it using the default Cedar Backup log
file, etc. You only need to use the switches if you need to change the
default behavior.
The main difference between Cedar Backup version 2 and Cedar
Backup version 3 is the targeted Python interpreter. For most users,
migration should be straightforward. See the discussion found at cback3(1)
or reference the Cedar Backup user guide.
- sourceDir
- The source directory on a local disk.
- s3BucketUrl
- The URL specifying the location of the Amazon S3 cloud storage bucket to
synchronize with, like s3://example.com-backup/subdir.
- -h, --help
- Display usage/help listing.
- -V, --version
- Display version information.
- -b, --verbose
- Print verbose output to the screen as well writing to the logfile. When
this option is enabled, most information that would normally be written to
the logfile will also be written to the screen.
- -l, --logfile
- Specify the path to an alternate logfile. The default logfile file is
/var/log/cback3.log.
- -o, --owner
- Specify the ownership of the logfile, in the form user:group. The default
ownership is root:adm, to match the Debian standard for most
logfiles. This value will only be used when creating a new logfile. If the
logfile already exists when the cback3 script is executed, it will retain
its existing ownership and mode. Only user and group names may be used,
not numeric uid and gid values.
- -m, --mode
- Specify the permissions for the logfile, using the numeric mode as in
chmod(1). The default mode is 640 (-rw-r-----). This value will
only be used when creating a new logfile. If the logfile already exists
when the cback3 script is executed, it will retain its existing ownership
and mode.
- -O, --output
- Record some sub-command output to the logfile. When this option is
enabled, all output from system commands will be logged. This might be
useful for debugging or just for reference.
- -d, --debug
- Write debugging information to the logfile. This option produces a high
volume of output, and would generally only be needed when debugging a
problem. This option implies the --output option, as well.
- -s, --stack
- Dump a Python stack trace instead of swallowing exceptions. This forces
Cedar Backup to dump the entire Python stack trace associated with an
error, rather than just propagating last message it received back up to
the user interface. Under some circumstances, this is useful information
to include along with a bug report.
- -D,
--diagnostics
- Display runtime diagnostic information and then exit. This diagnostic
information is often useful when filing a bug report.
- -v,
--verifyOnly
- Only verify the S3 bucket contents against the directory on disk. Do not
make any changes to the S3 bucket or transfer any files. This is intended
as a quick check to see whether the sync is up-to-date. Although no files
are transferred, the tool will still execute the source filename encoding
check.
- -u,
--uploadOnly
- Implement a partial or "upload only" sync, instead of a full
synchronization. Normally, synchronization would remove files that exist
in S3 but do not exist in the directory on disk. With this flag, new files
are uploaded, but no files are removed in S3.
- -w,
--ignoreWarnings
- The AWS CLI S3 sync process is very picky about filename encoding. Files
that the Linux filesystem handles with no problems can cause problems in
S3 if the filename cannot be encoded properly in your configured locale.
As of this writing, filenames like this will cause the sync process to
abort without transferring all files as expected. To avoid confusion, the
tool tries to guess which files in the source directory will cause
problems, and refuses to execute the AWS CLI S3 sync if any problematic
files exist. If you'd rather proceed anyway, use this flag.
This command returns 0 (zero) upon normal completion, and several
other error codes related to particular errors.
- 1
- The Python interpreter version is < 3.7.
- 2
- Error processing command-line arguments.
- 3
- Error configuring logging.
- 5
- Backup was interrupted with a CTRL-C or similar.
- 6
- Other error during processing.
This tool is a wrapper over the Amazon AWS CLI interface found in
the aws(1) command. Specifically, cback3-amazons3-sync invokes "aws s3
sync" followed by "aws s3api list-objects".
Cedar Backup itself is designed to run as root. However,
cback3-amazons3-sync can be run safely as any user that is configured to use
the Amazon AWS CLI interface. The aws(1) command will be executed by the
same user which is executing cback3-amazons3-sync.
You must configure the AWS CLI interface to have a valid
connection to Amazon S3 infrastructure before using cback3-amazons3-sync.
For more information about how to accomplish this, see the Cedar Backup user
guide.
- /var/log/cback3.log - Default log file
If you find a bug, please report it.
If possible, give me the output from --diagnostics, all of the
error messages that the script printed into its log, and also any
stack-traces (exceptions) that Python printed. It would be even better if
you could tell me how to reproduce the problem, for instance by sending me
your configuration file.
Report bugs to <support@cedar-solutions.com> or via GitHub
issues tracker.
Written and maintained by Kenneth J. Pronovici
<pronovic@ieee.org> with contributions from others.
Copyright (c) 2004-2020 Kenneth J. Pronovici.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.