| s3cmd(1) | General Commands Manual | s3cmd(1) |
s3cmd - tool for managing Amazon S3 storage space and Amazon CloudFront content delivery network
s3cmd [OPTIONS] COMMAND [PARAMETERS]
s3cmd is a command line client for copying files to/from Amazon S3 (Simple Storage Service) and performing other related tasks, for instance creating and removing buckets, listing objects, etc.
s3cmd can do several actions specified by the following commands.
Commands for static WebSites configuration
Commands for CloudFront management
Some of the below specified options can have their default values set in s3cmd config file (by default $HOME/.s3cmd). As it's a simple text file feel free to open it with your favorite text editor and do any changes you like.
One of the most powerful commands of s3cmd is s3cmd sync used for synchronising complete directory trees to or from remote S3 storage. To some extent s3cmd put and s3cmd get share a similar behaviour with sync.
Basic usage common in backup scenarios is as simple as:
s3cmd sync /local/path/ s3://test-bucket/backup/
This command will find all files under /local/path directory and copy them to corresponding paths under s3://test-bucket/backup on the remote side. For example:
/local/path/file1.ext -> s3://bucket/backup/file1.ext /local/path/dir123/file2.bin -> s3://bucket/backup/dir123/file2.bin
However if the local path doesn't end with a slash the last directory's name is used on the remote side as well. Compare these with the previous example:
s3cmd sync /local/path s3://test-bucket/backup/will sync:
/local/path/file1.ext -> s3://bucket/backup/path/file1.ext /local/path/dir123/file2.bin -> s3://bucket/backup/path/dir123/file2.bin
To retrieve the files back from S3 use inverted syntax:
s3cmd sync s3://test-bucket/backup/ ~/restore/that will download files:
s3://bucket/backup/file1.ext -> ~/restore/file1.ext s3://bucket/backup/dir123/file2.bin -> ~/restore/dir123/file2.bin
Without the trailing slash on source the behaviour is similar to what has been demonstrated with upload:
s3cmd sync s3://test-bucket/backup ~/restore/will download the files as:
s3://bucket/backup/file1.ext -> ~/restore/backup/file1.ext s3://bucket/backup/dir123/file2.bin -> ~/restore/backup/dir123/file2.bin
All source file names, the bold ones above, are matched against exclude rules and those that match are then re-checked against include rules to see whether they should be excluded or kept in the source list.
For the purpose of --exclude and --include matching only the bold file names above are used. For instance only path/file1.ext is tested against the patterns, not /local/path/file1.ext
Both --exclude and --include work with shell-style wildcards (a.k.a. GLOB). For a greater flexibility s3cmd provides Regular-expression versions of the two exclude options named --rexclude and --rinclude. The options with ...-from suffix (eg --rinclude-from) expect a filename as an argument. Each line of such a file is treated as one pattern.
There is only one set of patterns built from all --(r)exclude(-from) options and similarly for include variant. Any file excluded with eg --exclude can be put back with a pattern found in --rinclude-from list.
Run s3cmd with --dry-run to verify that your rules work as expected. Use together with --debug get detailed information about matching file names against exclude and include rules.
For example to exclude all files with ".jpg" extension except those beginning with a number use:
--exclude '*.jpg' --rinclude '[0-9].*.jpg'
To exclude all files except "*.jpg" extension, use:
--exclude '*' --include '*.jpg'
To exclude local directory 'somedir', be sure to use a trailing forward slash, as such:
--exclude 'somedir/'
For the most up to date list of options run: s3cmd --help
For more info about usage, examples and other related info visit project
homepage at: http://s3tools.org
Written by Michal Ludvig and contributors
Preferred way to get support is our mailing list:
s3tools-general@lists.sourceforge.net
or visit the project homepage:
http://s3tools.org
Report bugs to s3tools-bugs@lists.sourceforge.net
Copyright © 2007-2015 TGRMN Software - http://www.tgrmn.com - and contributors
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.