vcheck - latest program version checker and auto-downloader
vcheck [options]
vcheck is a tool for checking for latest versions of
programs at HTTP and FTP locations given a list of URLs and (Perl-style)
regular expressions to match, and to optionally download them
automatically.
For a complete list of command line options, run
$ vcheck --help
vcheck's behavior can be influenced by both command line
options and a configuration file, which at the same time serves as its data
file, holding records of programs to check for. This config file is, by
default (see "FILES"), ~/.vcheck. It is structured
according to a syntax which is printed in detail when run as
$ vcheck --grammar
Details about both the grammar in general and the meaning of
involved keyword can be found in "GRAMMAR". An example of what a
config file looks like in principle can be found in
"EXAMPLES".
The basic purpose of vcheck is to check for new versions of
programs listed in its config file. The script is able to cope with all
kinds of common version numbers, including words like "pre" or
"alpha", etc. When a new version was found, the config file is
updated accordingly.
Furthermore, vcheck can be used to download files
automatically if a new version is/was found, and even delete obsolete
versions found locally automatically. A special field in each program's
record in the config file tells it which version has last been downloaded.
Where necessary or desired, the download can be disabled for specific
programs, or disabled in general and allowed in special cases. Besides, you
can specify preferences (both in general and on a per-program basis) as to
what kind of files to download if new versions are available, say, in
different formats.
vcheck also has features to limit the scope of programs to
check or download to a subset, such as defined by:
- a regular expression names have to match
- a minimal urgency (which can be defined on a per-program basis, as levels
of high, medium, and low)
- those programs which haven't yet been downloaded since a new latest
version was found
- those programs which previous queries failed for (optionally, a certain
minimum number of times)
- a conjunctive combination of several of these conditions
In case you're behind a firewall, an HTTP(-based) proxy can be
defined in a number of ways (precendence in this order):
- a specific HTTP or FTP proxy, respectively, defined in the config
file
- a common HTTP+FTP proxy defined in the config file
- a specific HTTP or FTP proxy, defined via the environment variables
$http_proxy/$HTTP_PROXY and
$ftp_proxy/$FTP_PROXY, respectively (each in this
order of precendence)
vcheck uses ANSI escape sequences to visually enhance its
output. Success messages are usually printed in green, error messages are
yellow or red (signalling severity). This feature can be disabled
temporarily by using the corresponding command line switch, or permanently,
via a setting in the configuration file (see "CONFIGURATION
SECION").
When run as
vcheck --grammar
vcheck will print its config file's grammar, i.e., the
formal structure of the entries therein. The individual fields' names are
printed along with short descriptions; details on their meaning and usage
can be found below in this section.
Per default (i.e., if the script's name has not been changed (see
"FILES") and if not overridden via
"--file"), vcheck reads its
configuration from ~/.vcheck. This file will also be rewritten
regularly whenever version information etc. about a program is updated. In
the course of such rewrites, entries will be sorted in a definable fashion,
and a hard-coded order of keywords and indentation scheme will be
applied.
Basically, the config file may contain two types of records: a
configuration section and any number of program sections. A record (or
section--these terms are used synonymously in this documentation) consists
of a keyword marking its beginning and a name (this only goes for program
sections), followed by an equal sign (`=') and a pair of curly braces
("{}"), between which the section's data is put.
Section data is a sequence of settings, or fields, of a number of
types, some of which are obligatory while others are optional, separated by
white space (typically, line feeds, to keep things readable). There are the
following types of fields:
- Boolean
- Keywords of this type set a property based on their mere presence. An
example of this is the config section field dldefaultno:
config = {
dldefaultno
}
- string
- String fields consist of a keyword followed by an equal sign (`=') and a
string representing the field's value. If the string value contains white
space or (double) quotation marks, it needs to be surrounded by (double)
quotation marks (`"'). In this case, both quotation marks inside the
string and backslashes need to be escaped by backslashes (`\'). Note that
string values may not span several lines but have to be contained on a
single one, and there may be validation rules as to what the value may be
like. Besides, string fields are typically required to be of non-zero
length.
An example of this type of field is the prog section
field comment:
prog foo = {
[...]
comment = Hello!
comment = "Comment with white space and \"quotes\"!"
[...]
}
- string enumeration
- String enumerations are basically string fields with but a limited set of
allowed values. An example of this is the prog section field
dl, whose value must be either "yes" or "no",
if present:
prog foo = {
[...]
dl = yes
[...]
}
CONFIGURATION SECTION
The configuration section is optional and, if present, contains
settings globally affecting vcheck's default behavior. The
configuration section is unique per file (although multiple occurrences with
non-conflicting settings are allowed, but these will be joined into a single
section once the file is rewritten).
The keyword introducing a configuration section is config.
Thus, a config section's principal layout looks like this:
config = {
[...]
}
The keywords allowed inside ("[...]") the config
section are explained in detail below (listed in alphabetical order):
- defaulturgency
(enumeration: high, medium, low)
- Specifies the checking urgency level to assume, unless specified otherwise
in a program's record via prog.urgency. Urgencies allow for a crude
selection of programs to check for via the
"--urgency" command line parameter. In
absence of this option, the default urgency is medium.
- deleteold
(Boolean)
- If included in the config section, causes the script to
automatically look for and delete versions of a program obsoleted by a new
download. May be overridden by prog.deleteold. See the latter
for details.
Special note: Use at your own
risk!
- dldefaultno
(Boolean)
- By default, don't download. This causes the script to download only those
programs whose dl option is explicitly set to yes when run
with the "--download" parameter.
- dldir (string:
absolute directory path)
- This option specifies an absolute path (i.e., relative to the root
directory) of a directory where to put downloaded files. If the download
directory isn't set via this or even more explicitly via a
prog.dldir option, downloads will end up in that directory in which
the script is executing.
- dlexec
(string)
- Specifies a command to be executed after any successful download (unless
overridden for a particular program via prog.dlexec). A successful
download in this context is one whose file type has been recognized and
whose integrity could be confirmed. In unizoid environments, the command
is executed under whatever shell the environment variable
$SHELL defines.
The command string is subject to expansion of the following
placeholders (see "PLACEHOLDERS" for their meaning):
"__DLURL__",
"__FILE__",
"__NEWVER__",
"__PROG__",
"__RAWVER__",
"__URL__". Additionally, `~/' will be
replaced by the user's home directory.
config.dlexec may prove useful to, e.g., automatically
convert, say, gzipped to bzipped files using a helper script, or to log
downloads (see "HINTS").
- dlprefs
(string)
- A semicolon- (`;'-) separated list of Perl-style regular expressions
defining download preferences. Each of the regular expressions is supposed
to match a particular file type that's possible or likely to be
encountered. The order in which the expressions occur defines their
precedence (the first matching expression will determine which of a set of
available file types of a given program version will be selected for
download). This value is the default in effect unless specific preferences
are defined on a per-program basis using prog.dlprefs. If neither
config.dlprefs nor prog.dlprefs is set, the file to be
downloaded is chosen pseudo-randomly, if multiple pattern matches occur.
For these download preferences to make any sense, file- and
version-matching expressions need to be sufficiently non-restrictive to
match several possible extensions. For example,
"foo-("__VER__")\\.t" will
match both ".tar.gz" and ".tar.bz2" files, and
setting dlprefs to "\\.tar\\.bz2$;\\.tar\\.gz$" will
cause the script to preferrably download ".tar.bz2" files.
- dlretry (string:
non-negative integer number)
- The number of times to retry downloading after a failed download. If this
option isn't specified, the number of retries defaults to 0. A retry is
considered to have failed if either the connection failed, the retrieved
document was empty, or the file type has been recognized and its integrity
verified.
- eagerquote
(Boolean)
- If this option is set, all string parameters of configuration file options
will be surrounded by double quotes. The default is to use quotes only
where necessary (e.g., for string parameters containing white space).
- echoexec
(Boolean)
- If this option is set, commands executed thanks to newverexec or
dlexec options will be echoed prior to execution.
- ftpproxy
(string: HTTP URL or "server:port")
- This option specifies a proxy to use for retrieving documents from FTP
locations. It specifies either the complete URL or the server and port (as
"server:port") of the proxy, and the proxy has to be a
HTTP-based FTP proxy. This option takes precedence over
config.proxy, if specified. If neither config.ftpproxy nor
config.proxy is set, the script uses the value the environment
variables $ftp_proxy or
$FTP_PROXY (in this order of precedence) are set
to, or no FTP proxy at all.
- httpproxy
(string: HTTP URL or "server:port")
- This option specifies a proxy to use for retrieving documents from HTTP
locations. It specifies either the complete URL or the server and port (as
"server:port") of the proxy. This option takes precedence over
config.proxy, if specified. If neither config.httpproxy nor
config.proxy is set, the script uses the value the environment
variables $http_proxy or
$HTTP_PROXY (in this order of precedence) are set
to, or no HTTP proxy at all.
- lastcheck
(string: date formatted as "YYYY-MM-DD HH:MM")
- The date and time the script was last run updating the configuration file.
This value is generated and updated automatically.
- newverexec
(string)
- A command to be executed whenever a new version of a program is found,
unless overridden on a per-program basis via prog.newverexec. The
command is executed under whatever shell the environment variable
$SHELL defines.
The command string is subject to expansion of the following
placeholders (see "PLACEHOLDERS" for their meaning):
"__NEWVER__",
"__PROG__",
"__RAWVER__",
"__URL__". Additionally, `~/' will be
replaced by the user's home directory.
- nocache
(Boolean)
- This option conserves some memory by not caching retrieved documents
(those fetched from prog.url locations). By default, the script
caches retrieved document so that program records referring to the same
web page won't result in (unnecessary) multiple retrievals during the same
session.
- plain
(Boolean)
- This option causes the script to generate plain (as opposed to
ANSI-enhanced) output by default. The option may be overridden by
specifying "--noplain" on the command
line.
- proxy (string: HTTP
URL or "server:port")
- This option specifies a proxy to use for retrieving documents from both
HTTP and FTP locations. It specifies either the complete URL or the server
and port (as "server:port") of the proxy. The proxy set via this
option may be overridden via config.ftpproxy and/or
config.httpproxy.
- sortby
(enumeration: name, url)
- This option specifies whether to sort prog entries by program name
(prog section identifier) or URL when rewriting the configuration
file. The default is to sort by name.
- xfersum
(Boolean)
- Corresponds to the command line option
"--xfersum". If set, the script will
print a total of the amount of data that has been received at exit. Can be
overridden via the command line switch
"--noxfersum".
- timeout (string:
non-negative integer number)
- The time (in seconds) after which attempted remote retrievals should be
aborted. The default is 90 seconds.
- verbose
(Boolean)
- If this option is set, the script will also print version numbers that
haven't been obsoleted. The default is to print only new versions (and
error messages). This setting can be overridden via the command line
switch "--noverbose".
PROGRAM SECTIONS
Program sections each define for a single program (package, ...)
an HTTP or FTP URL based on which the latest version of that program
available can be determined by vcheck using an additionally-defined
regular expression. There can (hypothetically) be any number of program
sections in a config file.
The keyword introducing a program section is prog. Each
prog section is identified by a unique identifier (there may not be
multiple prog sections with the same identifier). Thus, a
config section's principal layout looks like this:
prog Foo = {
[...]
}
The keywords allowed inside ("[...]") a prog
section are explained in detail below (listing in alphabetical order). All
fields are optional and allowed but once per prog section, unless
explicitly stated otherwise.
- An arbitrary comment string. If multiple such entries exist for a single
program record, their relative order will be maintained when rewriting the
configuration file.
- deleteold
(enumeration: yes, no)
- This option defines whether the script should look for and delete any
obsolete versions of a program located in its download directory after
each 'Usuccessful' download of a new version of that program. A successful
download in this context is any download of a file of a known type whose
integrity could be verified. Overrides config.deleteold; the
deletion of obsolete versions is disabled by default and only activated by
config.deleteold or prog.deleteold.
Any occurrence of prog.dlexplicit or
prog.dlintermediate in a program's record inhibits application of
deleteold for that program.
Special note: Use at your own
risk!
- disabled
(Boolean)
- This option causes the program record in question to be ignored (except
when the command line switch "--force"
is used).
- dl (enumeration:
yes, no)
- This option specifies whether to download the program in question when the
script is run with the "--download"
option. By default, a program will be downloaded when a new version is
found and the script is run with said parameter, unless
config.dldefaultno is set. prog.dl overrides the latter
option.
- dldir
(string)
- This option specifies a download directory on a per-program basis. If the
directory is absolute (i.e., relative to the root directory, as indicated
by a leading slash), it will be treated as an absolute path, otherwise it
will be considered relative to either config.dldir, if specified,
or the directory the script is executing in.
The dldir string is subject to expansion of the following
placeholders (see "PLACEHOLDERS" for their meaning):
"__NEWVER__".
- dldirlast
(string)
- This option specifies the download directory of the last downloaded
version of a program. It does not contains "PLACEHOLDERS" unlike
"prog.dldir". If the directory is
absolute (i.e., relative to the root directory, as indicated by a leading
slash), it will be treated as an absolute path, otherwise it will be
considered relative to either config.dldir, if specified, or the
directory the script is executing in.
This option is only used to store "PLACEHOLDER" free
dldir, it is overwriten at each new download.
- dlexec (string,
may be zero-length)
- Specifies a command to be executed after any successful download of the
program, overriding config.dlexec (if set). A successful download
in this context is one whose file type has been recognized and whose
integrity could be confirmed. The command is executed under whatever shell
the environment variable $SHELL is set to.
The command string is subject to expansion of the following
placeholders (see "PLACEHOLDERS" for their meaning):
"__DLURL__",
"__FILE__",
"__NEWVER__",
"__PROG__",
"__RAWVER__",
"__URL__". Additionally, `~/' will be
replaced by the user's home directory.
- dlexplicit
(string: HTTP or FTP URL; multiple allowed)
- Specifies an explicit download URL. Whenever a new version of the program
in question is found, the URL specified via this option will be downloaded
(if requested) instead of the one deduced from prog.url and
prog.regex.
The command string is subject to expansion of the following
placeholders (see "PLACEHOLDERS" for their meaning):
"__NEWVER__",
"__RAWVER__".
This option can also be used to, e.g., download multiple
packages on detection of a new version, provided that their names can be
specified. For an example of this, see "EXAMPLES".
- dlintermediate
(Boolean)
- If this option is set, intermediate versions (i.e., version referenced at
url newer than dlversion but older than the most recent
version available) will be downloaded as well if any are encountered when
a new version of the program is found. This option is useful for
downloading patches and suchlike, which depend on each other
consecutively. The default is to ignore intermediate versions.
- dlprefs
(string)
- A semicolon- (`;'-) separated list of Perl-style regular expressions
defining download preferences. Each of the regular expressions is supposed
to match a particular file type that's possible or likely to be
encountered. The order in which the expressions occur defines their
precedence (the first matching expression will determine which of a set of
available file types of a given program version will be selected for
download). This value overrides default preferences possibly defined via
config.dlprefs. If neither config.dlprefs nor
prog.dlprefs is set, the file to be downloaded is chosen
pseudo-randomly, if multiple pattern matches occur.
For these download preferences to make any sense, file- and
version-matching expressions need to be sufficiently non-restrictive to
match several possible extensions. For example,
"foo-("__VER__")\\.t" will
match both ".tar.gz" and ".tar.bz2" files, and
setting `dlprefs' to "\\.tar\\.bz2$;\\.tar\\.gz$" will cause
the script to preferrably download ".tar.bz2" files.
- dlreferrer
(string, may be zero-length)
- Specifies an HTTP referrer to use when downloading a program package. By
default, the version-determining document (i.e., the last url
value, with placeholders expanded) is used.
- dlversion
(string)
- This parameter stores the last downloaded version of the program in
question and is updated whenever a new version is found (except when
running in read-only mode). If prog.transform is set, the stored
version will have been transformed from the one matched by
prog.regex.
- errors (string:
non-negative integer number)
- This field stores the number of errors during version checks and is reset
once a check succeeds. A high value of this field is indicative of an
outdated URL or file name matching regular expression and will be remarked
upon by the script. Additionally, it is possible to limit the scope of an
operation to erroneous records via the
"--errors" command line parameter.
- lastcheck
(string: date in format "YYYY-MM-DD HH:MM")
- This field stores the date and time the program in question was last
checked (no matter whether successfully or unsuccessfully).
- newverexec
(string, may be zero-length)
- A command to be executed whenever a new version of a program is found,
overriding a possible definition via config.newverexec. The command
is executed under whatever shell the environment variable
$SHELL defines.
The command string is subject to expansion of the following
placeholders (see "PLACEHOLDERS" for their meaning):
"__NEWVER__",
"__PROG__",
"__RAWVER__",
"__URL__". Additionally, `~/' will be
replaced by the user's home directory.
- regex (string;
required; multiple allowed)
- This required field is supposed to contain a Perl-style regular
expression matching desired versions of the program in question given the
document at prog.url as input. Note that the regexp needn't match
the complete file name--when considering a download, the script will
auto-expand the match as seen fit.
Regular expressions for matching programs' version numbers
have to be written in such a way that the $1
part (see the "perlre" man page), if the entire expression
matches, is exactly the version number. The option is subject to
placeholder expansion: "__VER__" will
be replaced by a pre-manufactured (non-greedy) regular expression
matching version numbers compliant with any of a number of common
schemes. Note that in order to yield a $1 match
as required, "__VER__" still needs to
be put in parentheses. For examples of prog.regex values, see
"EXAMPLES".
In order to cope with particularly complex remote scenarios
(such as version-dependent directory hierarchies), multiple url,
regex, and transform fields may be specified per program.
In this case, the script will match urls and regexes
starting with the first and continuously proceeding to the next field of
each type (in sync, as long as both of them are available, or using the
last one available otherwise) and match the regexp against the
corresponding document. In order for this to be of any use, the second
(and each potential later) url will have to contain a
"__NEWVER__" or
"__RAWVER__" placeholder (see
"PLACEHOLDERS") which will be replaced by the previously
matched latest [transformed] version (the same substitution is done for
regex). The version that will finally be considered the latest
for the program in question will be the one determined by matching the
last regex against the last urls document. For an example
of how this can be used in practice, see "EXAMPLES".
Possible multiple transform fields will be processed in
sync with the respective url and regex fields as long as
additional transform fields are specified. If there are more
url and/or regex fields than transform fields, the
last-specified transform expression will be used for further
iterations. If, on the other hand, there are more transform than
url/regex fields, further retrievals/matches will be done
based on the last url/regex. The author has, however, no
idea how this could be of any use.
When the config file is rewritten, multiple url,
regex, and/or transform fields will be interleaved to
facilitate comprehension and retain their relative order.
- tranform
(string; multiple allowed)
- A Perl expression transforming a version number in
$_ (obtained by a prog.regex match) in some
way the user deems adequate. For examples of how this might come in handy,
see "EXAMPLES". The return value of the code fragment, i.e., the
value of its last expression, is used as the transformed version and will
henceforth be the basis for version comparison for the program in
question.
- urgency
(enumeration: high, medium, low)
- Defines the urgency with which to check for the specified program.
Urgencies allow for a crude selection of programs to check for via the
"--urgency" command line parameter. If
there is no urgency defined, it defaults to either
config.defaulturgency (if set) or medium.
- url (string: HTTP or FTP
URL; required; multiple allowed)
- This required field defines the HTTP or FTP URL to retrieve as the
document to scan for in order to detect the availability of new program
versions by matching against prog.regex. Note that if the URL is a
directory (especially, an FTP directory which is supposed to be listed),
the URL needs to end in a slash (`/'). If the target document is an
HTML page, its source code will be matched against prog.regex,
aiming at links embedded in the document. An alternate download URL can be
specified via prog.dlexplicit.
In order to cope with particularly complex remote scenarios
(such as version-dependent directory hierarchies), multiple url,
regex, and transform fields may be specified per program.
In this case, the script will match urls and regexes
starting with the first and continuously proceeding to the next field of
each type (in sync, as long as both of them are available, or using the
last one available otherwise) and match the regexp against the
corresponding document. In order for this to be of any use, the second
(and each potential later) url will have to contain a
"__NEWVER__" or
"__RAWVER__" placeholder (see
"PLACEHOLDERS") which will be replaced by the previously
matched latest [transformed] version (the same substitution is done for
regex). The version that will finally be considered the latest
for the program in question will be the one determined by matching the
last regex against the last urls document. For an example
of how this can be used in practice, see "EXAMPLES".
Possible multiple transform fields will be processed in
sync with the respective url and regex fields as long as
additional transform fields are specified. If there are more
url and/or regex fields than transform fields, the
last-specified transform expression will be used for further
iterations. If, on the other hand, there are more transform than
url/regex fields, further retrievals/matches will be done
based on the last url/regex. The author has, however, no
idea how this could be of any use.
When the config file is rewritten, multiple url,
regex, and/or transform fields will be interleaved and
retain their relative order.
- version
(string)
- Stores the latest known version of the program. In contrast to
prog.dlversion, this is the latest version detected, not the latest
version downloaded. If a prog.transform option is set, the stored
version will have been transformed from the one matched by
prog.regex.
PLACEHOLDERS
In a number of string fields, certain placeholders are subject to
substitution by run-time values. These placeholders are (in alphabetical
order):
- "__DLURL__"
- The (file) URL from which the latest version of the respective program was
downloaded.
- "__FILE__"
- The local path to the respective latest-version download.
- "__NEWVER__"; "__NEWVER1__", "__NEWVER2__",
...
- "__NEWVER__" is replaced by the latest
transformed (or untransformed, if no transform expression is
in effect) version available as determined by the script.
When using multiple url/regex/transform
fields in order to cope with more complex remote site hierarchies,
"__NEWVER1__",
"__NEWVER2__", ... give access to
intermediately-determined versions. In this case,
"__NEWVER1__" is replaced by the
version matched by the first url/regex/transform
tuple, "__NEWVER2__" matches the
version matched by the second url/regex/transform
tuple, and so on.
- "__PROG__"
- The name (identifier) of the respective prog section.
- "__RAWVER__"; "__RAWVER1__", "__RAWVER2__",
...
- "__RAWVER__" is replaced by the latest
version available as determined by the script.
When using multiple url/regex fields in order to
cope with more complex remote site hierarchies,
"__RAWVER1__",
"__RAWVER2__", ... give access to
intermediately-determined versions. In this case,
"__RAWVER1__" is replaced by the
version matched by the first url/regex pair,
"__RAWVER2__" matches the version
matched by the second url/regex pair, and so on.
- "__URL__"
- The (last and expanded) URL used in order to determine the latest program
version.
- "__VER__"
- A pre-manufactured (non-greedy) regular expression matching version
numbers compliant with any of a number of common schemes.
- If you use Vim (version 5 or higher) as your editor, you can tell vcheck
to create a Vim syntax file providing syntax highlighting within the
editor by running the script as
vcheck --create-vim-syntax-file
If you wish to have Vim apply the syntax rules automatically
when editing "~/.vcheck", add this line:
au BufEnter */.vcheck so $VIM/syntax/vcheck.vim
or, alternatively, one with an explicit path:
au BufEnter */.vcheck so /path/to/syntax/vcheck.vim
to your "~/.vimrc" and substitute an appropriate
path. Of course you need to as well be sure to copy the file into the
designated directory.
- It's no problem to just check for new versions by default and run
vcheck again afterwards to download updated packages. Running the
script as
vcheck -dc
or
vcheck --download --catch-up
respectively, will try to download only those files whose
latest downloaded version has been knowingly obsoleted, without checking
again for new versions of all other programs.
- To check only those program locations that failed during the latest
attempt(s), run
vcheck -e
or
vcheck --errors
respectively.
- If you add a line
dlretry = NUMBER
to your config file's config section, vcheck
will retry to download a file up to NUMBER times if it detects that it
was received incompletely. This will be the case if:
- the file has zero size
- the downloaded file's extension was recognized, and a check by the
respective decompressor etc. resulted in errors
- vcheck caches data retrieved from URLs (unless nocache is
set in the config file), so if you specify exactly the same URL for
different programs, this won't result in multiple retrievals, thus
improving efficiency.
- If you're curious to know how many program records have actually been
accumulated in your config file over time, run vcheck as
vcheck --syntax
This will check the config file's syntax and, as a
side-effect, print the number of programs registered.
- Even if you know from some other source that there is a new version
of a program vcheck is configured for, you can still use that to
download the package. Just use its matching capabilities, e.g.:
vcheck -dm foo
- If one of your records points to patches of some program, and you want to
make sure you won't miss an intermediate one when downloading (and suppose
you don't run vcheck in download mode too frequently), you can add
the boolean field dlintermediate to the respective program's
section in the config file, and vcheck will try to download all
versions newer than dlversion. Note that in those circumstances,
dlversion is set to the latest (intermediate) version the download
attempt succeeded for (which means that, if, say, versions 1 through 3 are
to be downloaded and all downloads except that of version 1 succeed,
dlversion will nevertheless be set to 3). A useful example for
this:
prog Linux/patches = {
dlintermediate
dlprefs = \.bz2$;\.gz$
dlversion = 2.3.6
regex = patch-(__VER__)\.[bg]z
url = ftp://ftp.kernel.org/pub/linux/kernel/v2.3/
version = 2.3.9
}
Supposing that 2.3.9 still is the latest version, running this
in download mode will retrieve Linux kernel patches 2.3.7 through 2.3.9,
*.bz2 preferred to *.gz (but accepting the latter if the
former is missing, rather than skipping the download entirely).
- There may be complex remote site structures, involving version-dependent
directory hierarchies, such as the layout used by the server for the AC
series of Linux kernel patches. The principal layout of that site looks
(or used to look, anyway) like this:
...
.../linux-2.4/2.4.8/patch-2.4.8-ac1.gz
.../linux-2.4/2.4.8/patch-2.4.8-ac2.gz
...
.../linux-2.4/2.4.9/patch-2.4.9-ac1.gz
.../linux-2.4/2.4.9/patch-2.4.9-ac2.gz
...
The problem here is that the bottom-level directory's name
varies depending on the regular Linux version an AC patch is based on.
The way to deal with this most conveniently in vcheck looks like
this:
prog Linux/patch/AC = {
dlintermediate
url = http://www.kernel.org/.../linux-2.4/
regex = (\d+\.\d+\.\d+)
url = http://www.kernel.org/.../linux-2.4/__NEWVER__/
regex = patch-(__VER__-ac\d+)\.gz
}
(Note that the URLs have been abbreviated for the sake of
readability.) This kind of configuration will cause vcheck to
start by retrieving the first url field's document and match the
first regex against it. It will then proceed with the second
url field's document, matching it against the second
regex, replacing its __NEWVER__ placeholder by the latest
version previously matched. The version finally determined as the
current version for the program record is the one determined by the last
match.
On a side note, version numbers determined during matches
further back than the previous one can be accessed via delimiters of the
format __NEWVER#__, where `#' is a number indicating the number
(1..) of the url/regex pair's version match it should be
replaced by. For more details on the mechanism, see the descriptions of
url and regex in "PROGRAM SECTIONS", and
"PLACEHOLDERS".
Regarding the example, it is left to the user to figure out
how to extend the record to even automatically cope with changes to the
Linux kernel's minor version.
":-)"
Here's another example of a three-level hierarchy, which used
to fit the GIMP's site layout at one point in time:
prog GIMP/devel/patch = {
comment = "Will download complete package if no patch available."
dlprefs = patch-.*?\.bz2$;patch-.*?\.gz$;gimp-.*?\.bz2$;gimp-.*?\.gz$
url = ftp://ftp.gimp.org/pub/gimp/
regex = (?<!\w)v(__VER__)
url = ftp://ftp.gimp.org/pub/gimp/v__NEWVER__/
regex = (?<!\w)v(__VER__)
url = ftp://ftp.gimp.org/pub/gimp/v__NEWVER1__/v__NEWVER__/
regex = (?:patch|gimp)-(__VER__)\.[bgt]
}
- If you want to retrieve some program whose version is, say, a date in
format "dd-mm-yyyy", this will be misinterpreted by the version
comparator because the most significant sub-"version" isn't the
initial one. You can work around this by specifying some Perl expression
transforming the original version in the respective program's section,
such as:
transform = "s/(\d+)-(\d+)-(\d+)/$3-$2-$1/; $_"
This piece of code is given the respective version in
$_, and after its evaluation, vcheck
replaces the original value by what the eval() returns.
Alternatively, this would achieve the same:
transform = "join '-', reverse split /-/, $_"
- Some sites use redirection scripts for download URLs. Consider a situation
where a downloads page lists available packages of a program, with links
pointing to some server-side script referring your browser to some URL
which is in turn redirected by HTTP means to a final file URL (the
PHP site, for example, used to make use of this obscure scheme at one
time). The way to cope with this in vcheck consists in using a
dlexplicit field like this:
prog PHP = {
dlexplicit = http://www.php.net/distributions/php-__VER__.tar.gz
regex = php-(__VER__)\.t
url = http://www.php.net/downloads.php
}
Effectively, this will use the actual url field only to
determine the current version and then paste it into a pattern of the
corresponding download URL, thus bypassing the redirections. The obvious
disadvantage of this feature consists in its increased dependency on
server-side access structures.
- Suppose you're interested in some program distributed via more than one
package (such as Vim, which is split into a source and a run-time
package). The means vcheck provides to cope with this once again is
the dlexplicit option:
prog Vim = {
dlexplicit = ftp://ftp.vim.org/pub/editors/vim/unix/vim-__VER__-src.tar.gz
dlexplicit = ftp://ftp.vim.org/pub/editors/vim/unix/vim-__VER__-rt.tar.gz
regex = vim-(__VER__)(-src)?\.tar
url = ftp://ftp.vim.org/pub/editors/vim/unix/
}
- In order to have vcheck keep track of what has been downloaded (and
when), you might add something like this to your config file:
config = {
dlexec = "echo `date +%Y-%m-%d` '__PROG__' '__NEWVER__' >>~/.vchecklog"
}
Note however that program-specific dlexec will take
precedence over this setting.
- With a little creativity, vcheck can be used to check not only for
latest versions of programs or packages, but also web site updates and the
like. Also, the newverexec (see "GRAMMAR") field can be
used to pass a link to an external download tool if for some reason
vcheck's abilities prove insufficient for a particular
scenario.
Please make sure to read what's printed by vcheck when run
as
vcheck --help --grammar
as well as "GRAMMAR" before reading this section, to
learn about command line parameters and the configuration file's grammar.
Done so? Then read on...
Suppose there's a config file ~/.vcheck with the following
contents:
config = {
dlprefs = \.tar\.bz2$;\.(tar\.|t)gz$;\.zip$
lastcheck = "1999-06-21 08:15"
}
prog Foo = {
dl = no
errors = 2
regex = foo-(__VER__)\.tar
urgency = high
url = http://www.foo.org/pub/foo/
}
prog Bar = {
dlversion = 0.01beta
regex = (?i:bar-(__VER__)\.tar)
url = http://www.bar.org/bar/index.html
version = 0.01
}
prog Baz = {
regex = baz-(\d+)\.tar
urgency = low
url = ftp://ftp.baz.net/pub/source/
version = 123
}
First of all, you can deduce from this what date and time
vcheck was last run at with this config file. Trying to check for
Foo resulted in errors of some kind during the last 2 attempts, and
since there's no version field, it has presumably never been queried
successfully. Foo is never to be downloaded. Bar's latest
version as determined during one of the last checks was 0.01, but it wasn't
downloaded (0.01beta is the version of the last download). Finally,
Baz has never been downloaded (according to the config file, anyway).
As for downloads in general, *.tar.bz2 is preferred to
*.tar.gz and *.tgz, which in turn are more desirable than
*.zip files. If no target matching any of these extensions
case-insensitively is found, nothing will be downloaded.
Assume furthermore that the following references are currently
mentioned at the respective URLs of each program:
- for Foo:
http://www.foo.org/pub/foo/foo-3.14.tar.gz
http://www.foo.org/pub/foo/foo-3.14.tar.bz2
http://www.foo.org/pub/foo/foo-3.14a.tar.gz
http://www.foo.org/pub/foo/foo-3.14alpha.tar.gz
http://www.foo.org/pub/foo/Foo-4.0.tar.gz
- for Bar:
bar-0.01.zip
BAR-0.01.tar.bz2
- for Baz:
http://www.baz.net/pub/download/baz-124.rpm
Now let's discuss what some specific calls to vcheck, each
based on the above configuration, will result in. Again, for a complete list
of command line options (all short options have an equivalent long one), see
"`vcheck --help`".
- - "$ vcheck -n"
- This will check for all programs without updating the config file. It'll
report Foo 3.14 as new version (not 4.0, as regex doesn't
match this), as well as Baz 124.
- - "$ vcheck -d"
- This will check for all programs, report as above and try to download the
following file:
http://www.bar.org/bar/BAR-0.01.tar.bz2
Note that Baz 124 isn't among, because there wasn't a
link conforming to dlprefs, and downloads of Foo have been
disabled explicitly. The errors field of Foo is removed
since the check succeeded.
- - "$ vcheck -c"
- This will set dlversion = version for Bar and
Baz, without checking for the availability of new versions.
Effectively, this will prevent future calls to vcheck with
parameter ""-d"" from
downloading these files.
- - "$ vcheck -dc"
- This will step through all programs that downloads haven't been disabled
for in principle and whose dlversion is lower than version
(i.e., Bar and Baz in our example). For these, vcheck
will requery the respective sites to determine a download URL, and try to
download
http://www.bar.org/bar/BAR-0.01.tar.bz2
as in the above example.
- - "$ vcheck -m \!foo"
- will check for new versions of Bar and Baz. Note that you
may have to quote the leading exclamation mark as well as some characters
used in regular expressions specified on the command line, in order to
prevent your shell from interpreting them.
- - "$ check -u medium -m b"
- will check only for Bar, as it is the only program whose
urgency is at least medium and whose name contains a
`b'.
- - "$ vcheck -e"
- will check only for Foo, since checking for that failed
previously.
- vcheck, the script itself
- ~/.vcheck, its configuration file
In fact, vcheck doesn't look for a config file
~/.vcheck, but for one of the same name as the script (with a
possible extension stripped off). So if you rename the script to, say,
foo.pl and run it, it'll try to open ~/.foo.
- ~/.vcheck.lock, lock file created when not running in read-only
mode
Actually, the file's name is that of the config file with an
extension of .lock added.
- $http_proxy/$HTTP_PROXY and
$ftp_proxy/$FTP_PROXY, each in this order of
precedence, specify the HTTP and/or (HTTP-based) FTP proxy to use, unless
overridden. The format is either "server:port" or a complete
URL.
- $HOME, the current user's home directory
- $SHELL, used by Perl in unizoid environments when
executing helper applications
- add option to config section allowing for the dlexec entry to be
"inherited" from the config section rather than be overridden by
per-program dlexecs (also define order of execution!)
- check behavior if an HTTP url's download link references a
different target base directory
- make "--list" not rewrite the config file, thus allowing for it
to be run in parallel to another instance
- add an option to re-download the latest version (if local file doesn't
exist)?
- code clean-up: array used for download specifications -> hash
- determine and describe way of reliably matching directories consisting of
but a version number on an FTP server independently of whether the page in
question is received by proxy or without one
- scenario: link description contains version, download link entirely
differently named
- separate "--force" options for
overriding disabled, dl?
- max download size command line parameter?
- extend Vim syntax file generation to highlight placeholders in string
variables' values?
- follow HTTP redirections
- evaluate HTTP headers after retrievals
- resume downloads?
- XMLize the config file format???
- make it multi-threaded???
- All output is currently printed on STDOUT.
- Placeholders are used but no way of escaping literal occurrences of those
strings is provided.
- dlprefs uses semicolons as delimiters, but there's no way of
escaping them if they are meant as a part of one of the regular
expressions.
- There's presumably little to do in order to get vcheck to run in
Microsoft Windows. One issue worth noting is that directories (such as
dldir values) are expecedted to use unizoid delimiters (i.e.,
slashes (`/'))--this should perhaps be revised to be portable.
Mail bug reports to the author.
vcheck is copyright (c) 1999-2001 by Marco Goetze,
<gomar@mindless.com>. It is distributed under the terms of the
Artistic License, a copy of which is included with the script's
distribution. Use at your own risk.