apt-move - move cache of Debian packages into a mirror
hierarchy.
apt-move [-c conffile] [-d
dist] [-afqt] command
The apt-move script is used to move a collection of Debian
package files into a proper archive hierarchy of the form
$LOCALDIR/pool/... where LOCALDIR is specified in the
configuration file. It is intended as a tool to help manage the
apt-get(8) file cache, but could be configured to work with any
collection of Debian packages.
Additionally, using the sync and mirror commands,
you can build your own local mirror of portions of a selected binary and/or
source distribution.
Running apt-move periodically will assist in managing the
resulting partial mirror by (optionally) removing obsolete packages and
creating valid local Packages.gz and Sources.gz.
The following commands are accepted by apt-move:
- get [ dir
]
- This generates the master files using Packages and Sources files from the
apt(8) cache. The master files are used to keep track of what
packages are available, and where packages should be installed. If
dir is specified, it will be used in lieu of the LISTSTATE
variable.
- getlocal [ dir
]
- This is an alias of get. It may be removed in future releases.
- move
- Moves a collection of packages into the local mirror tree. Uses existing
master files (see get) to repair any mangling done to the package
names. Any packages that aren't listed in the master files or are obsolete
will be left in the file cache directory. Obsolete packages will also be
copied into the archive but they will be removed after the next delete
operation. In the these two cases, the package is considered to have been
skipped.
- delete
- Delete obsolete package files. Configurable through the DELETE and
MAXDELETE settings in the /etc/apt-move.conf file (see the
CONFIGURATION section below). It also deletes any index files of
packages that are no longer in the archive. This means that you can simply
delete packages from the archive without affecting its consistency as long
as you run the delete command afterwards.
- packages
- Builds new local versions of Packages.gz and Sources.gz files.
- fsck
- Rebuilds all index files used to make Packages and Sources files and
reprocess all packages in the archive by calling movefile on them.
Use this if you are upgrading from an old version (<< 4.2) of
apt-move.
- update
- This is an alias, equivalent to 'get move delete
packages'. This is the preferred method for moving package files
from your cache into a local mirror.
- local
- This is an alias, equivalent to 'move delete
packages'.
- localupdate
- This is an alias for update. It may be removed in future releases.
- mirror
- This command automatically runs get, then uses
/usr/lib/apt-move/fetch and apt-get(8) to download any
packages missing from your mirror. The downloaded files will be installed
into the repository using move. Finally, it runs packages
and exits. See the DIST and PKGTYPE settings in
/etc/apt-move.conf. Before using this command, you need to set up a
$LOCALDIR/.exclude file containing patterns to exclude unwanted
files from your mirror. See the SAMPLE.exclude file for an example.
See also the Exclude file section of NOTES below. Note that
this command will only mirror packages for the architecture that you are
running on. It will, however, mirror all source packages.
- sync
- Similar to the mirror function, but only gets the packages that are
currently installed on your system. It uses dpkg(8)
--get-selections to find out what files to download. It will skip
any files that match one of the patterns in the $LOCALDIR/.exclude
file (if it exists). sync will get the latest versions of the
packages, regardless of the version currently installed on your system
(think about it).
- exclude
- This command is used to test your $LOCALDIR/.exclude pattern file.
It will go through the master lists and print any file that matches one of
the patters in $LOCALDIR/.exclude. This will show you exactly what
files you have EXCLUDED from your mirror. The -t (test) flag has no
affect on this command. This uses your existing master files, and does not
require an internet connection.
- movefile files...
- This command is similar to move. Instead of moving files from
FILECACHE, it will move the files specified on the command line.
- copydir directory...
- This command is similar to movefile. Instead of moving files from the
command line, it will copy .deb files from the specified directory. This
is useful for copying the pool of a Debian CD to an apt repository.
- listbin [
mirror | sync | repo ]
- This command prints a list of packages which may serve as the input to
mirrorbin or mirrorsrc. If the argument is mirror or sync,
it will produce the same lists that the mirror and sync
commands use. If the argument is repo, the list produced will
contain the packages that are currently in the apt-move repository.
- mirrorbin
- This command will fetch the list of packages specified on the standard
input, and place them into the archive in the same way as mirror
does.
- mirrorsrc
- This commands acts like mirrorbin, except that it fetches source
packages instead of binary ones.
The following options are available from the command line:
- -a
- Process all packages. By default, commands like listbin and
mirrorbin only process packages that differ in version between the
apt-move repository and the archive being mirrored. This option causes all
packages to be considered even if the apt-move repository already contains
the latest version.
- -c conffile
- Use conffile as the configuration file instead of
/etc/apt-move.conf.
- -d dist
- Use dist as the default suite instead of the value of DIST
from the configuration file.
- -f
- Forces deletion of files even when the percentage of files to delete
exceeds the MAXDELETE setting. This is useful if apt-move
aborts with an error saying that too many files would be deleted, and you
want to delete the files anyway. (Use with caution.) If you get this
error, using -ft will show you the complete list of files, so you
can verify them before you use -f.
- -q
- Suppresses normal output. This option is useful when apt-move is
used in a non-interactive script.
- -t
- Makes a 'test run' and reports what WOULD be done for option but
does not modify any of the cache or mirror files.
Before using apt-move, edit the /etc/apt-move.conf
file to match your local setup. Always remember to use the test
parameter after any change in your configuration to make sure it will work
like you want it to. You may also want to set the DELETE option to
no to turn off file deletes until everything else is working
successfully.
The file is read as a Bourne shell script. So you must obey the
syntactical rules of sh(1). In particular, values with spaces in them
must be quoted with single or double quotes.
The following settings are recognized by apt-move (shown
here with their defaults):
- APTSITES="debian.midco.net
non-us.debian.org"
- Set this to the names of sites in your /etc/apt/sources.list that
you wish to mirror. The value /all/ refers to all non-file URIs.
- LOCALDIR=/mirrors/debian
- This is the full (absolute) path to your debian directory (the top of your
local mirror).
- DIST=stable
- This is the default suite assigned to packages when the Release
file is missing. It does not have any effect on whether a suite is stored
in the archive.
- PKGTYPE=binary
- Set this to your choice of: binary, source or both to
tell the mirror, sync and movefile which type(s) of
files to get.
- FILECACHE=/var/cache/apt/archives
- The directory where your local cache of packages are. The default will
work for the apt-get(8) packages, unless you've changed the
configuration in /etc/apt/apt.conf.
- LISTSTATE=/var/lib/apt/lists
- The directory to your local cache of Packages files. The default will work
for the apt-get(8) packages, unless you've changed the
configuration in /etc/apt/apt.conf.
- DELETE=no
- Determines whether obsolete packages (packages not listed in the master
file, or packages that have been superceded in the repository) are to be
removed.
- MAXDELETE=20
- Maximum percentage of files apt-move is allowed to delete during a
normal run. Anything exceeding this will produce an error and abort the
script. I added this as a precaution so that you won't lose your entire
mirror when a new distribution is released. You can override this (with
caution) using the -f parameter with apt-move.
- COPYONLY=no
- If this is set to yes, then move and movefile will copy
instead of move. That is, the originals will be left alone.
- PKGCOMP=gzip
- This should be set to a space-separated list of compression formats that
apt-move should provide when generating Packages and Sources files. The
possible values are none, gzip and bzip2. With the
current apt package you should include at least none, as
otherwise apt will complain about missing files.
- CONTENTS=no
- If this is set to yes, then packages will generate Contents files.
- GPGKEY=
- If this is set to non-empty string, then packages will sign
generated Release files with the specified key. You must install
gnupg before enabling this option.
For the sync and mirror commands to function
correctly, you need to list your apt-move repository at the top of
/etc/apt/sources.list as a file URI.
- /usr/bin/apt-move
- The script.
- /etc/apt-move.conf
- Configuration file for the script.
- /usr/share/man/man8/apt-move.8.gz
- The manpage.
- /tmp/MOVE_*
- The temporary files created at runtime.
- /usr/lib/apt-move/fetch
- Utility to fetch files just like apt-get install -d. Except that no
dependency analysis is done.
- .apt-move/*.{binary, source}.local
- Put entries of local packages here. The fields are ``package priority
section source version task'' for the binary file, and ``package priority
section version'' for the source file. The version field may be set
to a single dash to refer to the latest version in the archive. Blank
lines and lines beginning with a hash are ignored.
The mirror command uses a file in the $LOCALDIR/
directory called '.exclude' which contains exclude patterns that are applied
against the files to be mirrored. These patterns were created with the
following limitation: they must work the same with with grep(1),
after any '*' characters are removed. Unless you're careful setting this up,
you'll get unexpected results. Run 'apt-move -t mirror'
first, to make sure you're getting the results you intended. Another way to
verify your exclude file is the use the exclude command for
apt-move to print a list of files your are excluding from your
mirror. See the sample
.exclude file (SAMPLE.exclude) for an example of an .exclude file.
The apt-move mirror and sync commands do not
test for available disk space. The current potato (main binary) distribution
is over 1Gb in size. Add the sources to that and it can eat up the space on
a partition really fast. I would advise you to put your mirror somewhere
other than the root partition. Set up your exclude file and run apt-move
-t mirror and examine the result.
Since apt-move gets the suite names from Release
files, which usually use the names stable, testing and
unstable, the suites in the repository are named accordingly. You can
simulate the code names by creating symbolic links in the dists
directory. For example, in order to make testing equivalent to
sarge, you could run ln -s testing $LOCALDIR/dists/sarge.
Alternatively, you could delete the testing subdirectory and run
ln -s sarge $LOCALDIR/dists/testing. This will cause future
executions of the get operation to use sarge whenever it sees
testing in the Release file.
apt-move may exit with one of the following error
messages:
- Could not read
configuration.
- apt-move could not find the /etc/apt-move.conf file. Run the
install script.
- Could not create
directory.
- For some reason, a necessary directory could not be created.
- You failed to select a
distribution.
- You did not configure a DIST setting in /etc/apt-move.conf.
- You specified an invalid
package type.
- You can only use binary, source or both for the
PKGTYPE setting.
- No master files
exist!
- You need to run apt-ve with the get command at least once in
order to create the master files which determine where packages are to be
installed.
- bc calculation returned invalid
result
- apt-move uses the bc(1) program to determine when the number
of files to delete will exceed the MAXDELETE setting in
apt-move.conf. If you get this error, make sure that
MAXDELETE is set to a number in the range of 1 to 100, without the
% sign. Otherwise you need to report this as a bug.
- Too many files to
delete!
- apt-move will report this error if the number of files to be
deleted exceeds the MAXDELETE setting in apt-move.conf. You
need to study the output to determine if this is normal (in which case you
can override this using the force parameter), or if its due to some
drastic change on the mirror site (like a new release) or possibly due to
a partial download of the master Packages.gz or Sources.gz file.
- Your current mirror
directory is incompatible...
- You have just upgraded from an old version of apt-move. Update your
configuration, then run apt-move fsck and finally remove
.apt-move/ancient.
- Could not read
Release files.
- apt-move could not read the release files needed to build the
master files. Make sure you have run apt-get update and try again.
- Failed to remove
original files.
- apt-move could not remove the original copies of files that have
just entered the apt-move archive. Make sure that you have
permission to delete those files.
- Please remove
$LOCALDIR/backup.
- You must remove LOCALDIR/backup before running the fsck
command.
- Unknown DIST
setting.
- The value of DIST must match the Archive field in the
Release file of the distribution that you are trying to mirror.
- Cannot find index files
for APTSITES.
- apt-move could not find any index files for the get
operation. You should either run apt-get update, or run
apt-move with -f if only the *.local files have been
changed.
The exclusion system was designed prior to the introduction to the
package pools. Hence its content still relates to the old structure of the
Debian archive. This is confusing and it should be replaced with a new
exclusion system.
The DIST variable does not control what is actually
downloaded by the fetch utility. It should have a similar effect to
that of apt-get -t.
Michael Merten <mikemerten@yahoo.com>
Herbert Xu <herbert@debian.org>