nnadmin - nn database administration
nnadmin is a control program for the nnmaster(1M)
daemon which is responsible for building and maintaining the database used
by the nn(1) news reader.
nnadmin allows you to display extracts from the log file,
display the "raw" contents of the database, make consistency
checks on the database, instruct the running nnmaster to expire one
or more groups, alter the options of the running nnmaster, and much
more.
nnadmin runs in two modes: interactive and
non-interactive.
In interactive mode, simple one line menus are used to show the
available operations which are then selected by typing the letter associated
with the command (normally the first letter in the command name).
In non-interactive mode, the commands argument will be used
as a series of key-strokes which are interpreted exactly as if they were
typed in from the keyboard in interactive mode. For example, to stop the
nnmaster, the following invocation of nnadmin can be used:
nnadmin MK
which will select the (M)aster submenu from the main menu, and then the (K)ill
entry from the submenu.
In non-interactive mode, the menus are not displayed and the
commands are not echoed! nnadmin will exit when there are no more
key-strokes to be read from the commands argument. It is not possible
to specify a group name in the commands argument, so the
functionalities of nnadmin that relates to specific groups are only
available in interactive mode.
Some "dangerous" commands will require that you confirm
them by following them by "Y" on the command line. The most
noteable are IY (initialize database) and EY (expire all groups). These
commands will be marked with a [Y] following the command name.
You can also invoke an interactive nnadmin using the
:admin command in nn.
At all prompts you can hit `!' to spawn a subshell.
The working directory of the subshell will be changed to the
database directory when invoked from the MASTER or DUMP menus, and it will
changed to the group's spool directory (if it exists) when invoked from the
GROUP menu.
From the main menu (identified by the ADMIN prompt) you can
select the following operations:
- C)onf
-
Show current configuration parameters such as directories, files, programs,
network usage, etc.
- E)xpire [Y]
-
Send a request to the nnmaster daemon to schedule (and run) expire
for all groups in the database.
- G)roups
-
Enter the GROUP submenu.
- I)nit [Y]
-
Send a request to the nnmaster daemon to recollect all groups in the
database.
- L)og
-
Enter the LOG submenu.
- M)aster
-
Enter the MASTER submenu.
- Q)uit
-
Quit nnadmin.
- S)tat
-
Print general statistics about the database. See the section on Database
Statistics below.
- U)pdate
-
Update the incore copy of the database master index.
- V)alidate
-
Make a thorough consistency check on the database. If inconsistencies are
found in a group, you will be asked whether a request should be sent to
the nnmaster daemon to recollect the group (in non-interactive
mode, requests will be sent automatically for all corrupted groups).
- W)akeup
-
Send a wakeup signal to the nnmaster daemon to have it receive
messages sent to it, perform the required actions, and then collect
articles as necessary.
- Z (silent
validation)
-
This operation is identical to the Validate operation, expect that no output
is produced during the consistency check; this operation is used by the
nnmaster to execute the -C option.
The master menu (identified by the MASTER prompt) provides
access to overall database information, and to send control messages to the
nnmaster daemon.
- C)heck
- In interactive mode and in verbose batch mode (nnadmin MC), print a
message telling whether nnmaster is running or not. In silent batch
mode (nnadmin =MC) exit with a status code of 0 if nnmaster
is running and 1 otherwise; this may be useful is administrative
scripts.
- D)ump
- Enter the DUMP submenu.
- F)iles
-
Print a listing (using ls(1)) of all the data and index files in the
database.
- G)roup
-
Print the master index entry for a single group identified by its internal
group number.
- K)ill
-
Stop the nnmaster when it has finished its current task.
- O)ptions
-
Change the runtime options of the running nnmaster daemon. Currently,
only the value of the -r and -e options can be modified.
- S)tat
-
Print general statistics about the database. See the section on Database
Statistics below.
- T)race
-
Turn the trace option -t on or off in the running nnmaster.
The dump menu (identified by the DUMP prompt) allows you to
print the master index entry for various selections of groups in the
database.
- A)ll
-
Print all groups in the database.
- E)mpty
-
Print the empty groups in the database.
- H)oles
- Print the groups where the `min' field in the active file is not the first
article saved in the database (because it doesn't exist or because it is
ignored for some other reason, e.g. bad or old).
- I)gnored
- Print groups which are ignored, either in the GROUPS file or because of
some other condition (mainly no spool directory).
- N)on-empty
-
Print the non-empty groups in the database.
- V)alid
- Print the groups which are present in the active file.
- in(W)alid
- Print the groups in the database which are not present in the active
file.
The log menu (identified by the LOG prompt) enables you the
extract specific entries from the log file, and to truncate the log
file.
The entries in the log file share the following format:
<class>: <date> <time> (<user>):
<message>
where <class> identifies the message class, the <date>
and <time> specify when the entry was made, the <user> specifies
who created the entry (the letter "M" denote the nnmaster),
and the <message> is the text of the entry.
To extract the log file entries of a specific class, simply enter
the letter identifying the class:
- A - admin to master
communication
-
This class of messages are related to the sending of messages from an
nnadmin program to the nnmaster daemon.
- B - bad articles
- Reports about bad articles which have been ignored or removed (controlled
by the -b and -B options to nnmaster).
- C - collection
statistics
-
Statistics about collection of new articles. The message has the format:
Collect: nnn art, ppp gr, ttt s
meaning that nnn articles in ppp groups were collected in
ttt seconds (real time).
- E - fatal errors
-
Fatal errors encountered during operation. These errors require manual
intervention to be fixed (some of the fatal errors occur if thing that
"cannot happen" happens anyway, and may indicate a bug in the
software).
- M - nnmaster
messages.
-
Master start/stop messages.
- N - NNTP related
messages
-
Various messages related to the NNTP part of the nnmaster, mostly about lost
connections and failed attempts to connect to the NNTP server. These
messages should only appear if you use NNTP, and your NNTP server is down
for some reason.
- O - old articles
- Reports related to ignoring (and removing) old articles when building the
database (controlled by the -O and -B options to
nnmaster).
- R - reports
-
Non-fatal error which enables the nnmaster to continue operation, but
may prevent a user to run nn (file access problems). Reported
problems should be checked. The most common report message will probably
be
some.group: no directory
which indicates that the spool directory for that group has disappeared
(most likely because it has been rmgroup'ed).
- T - trace output
-
Messages produced as a result of using the -t option on the nnmaster.
This is primarily for debugging purposes.
- U - usage statistics
-
If nn is compiled with the STATISTICS option enabled, an entry will
be made in the log file every time a user has spent more than five minutes
on news reading. The message will have the following format:
USAGE hours.minutes
Since it is possible to suspend nn, or leave the terminal while
nn is active, nn tries to be intelligent when it calculates
the usage time so it will reflect the actual time spent on news reading.
The usage statistics can be summarized using the nnusage(1M)
program.
- V - validation
errors
-
When inconsistencies are detected in the database during validation, an
entry for each corrupted group will be entered in the log file.
- X - expire
statistics
-
Messages similar to the Collect statistics reporting the result of running
expire on the database. Reports related to ignoring, removing,
renumbering, and reactivation of groups are also given class X.
To extract a specific entry class, grep(1) is used, so it
may take a while on a large log file.
There are also a few special operations on the log file:
- G)roup
-
Extract the entries which refers to a specified group.
- (1-9) tail
-
Invoke tail(1) to extract the last 10-90 entries in the log
file.
- space
-
Equivalent to 1 (list last 10 lines of log).
- (.) all
-
Display the complete log file.
- (@) clean [Y]
-
Move the Log file to Log.old, and create a new empty Log file. If you want
to clean out the old log file as well, simply repeat the clean operation
(this will result in an empty Log.old file.)
When you enter the group menu (identified by the GROUP
prompt), nnadmin will prompt you for the name of a news group, which
you can enter with the usual completion feature described in the
nn(1) manual. You can then perform the following operations on the
specified group:
- C)lear_flag
-
Clear a group specific flag. See the section on group flags below.
- D)ata
-
Dump the contents of the data file containing the extracted article headers
for the group.
- E)xpire
-
Request the nnmaster to run expire on the group.
- F)iles
-
List the files (using ls(1)) containing the index and data for the
group.
- G)roup
-
Switch to another group.
- H)eader
-
Dump the master index entry for the group.
- R)ecollect
-
Request the nnmaster to recollect all articles in the group.
- S)et_flag
-
Set a group specific flag. See the section on group flags below.
- V)alidate
-
Perform validation on the group's database information.
- Z)ap [Y]
-
Remove group from news system - this will be done by running the
rmgroup program which must reside in the NEWS_LIB directory. Of
course, this should be done with great caution.
You can set and clear the following flags for individual groups to
control the future behaviour of nnmaster on that group.
Notice that these flags will be reset to their default value if
you reinitialize the database using nnmaster -I. To change these
flags permanently, they should be set or cleared in the GROUPS file.
- A)lways_digest
-
Normally, nnmaster will only attempt to split digests into individual
articles if it can easily recognize an article as a digest. This requires
that the word "digest" appears somewhere in the subject line,
and that one of the first few lines in the body of the article loosely
matches the subject. A few news groups frequently receives digests which
break one or both of these requirements. To have nnmaster split
these digests into individual articles anyway, you can turn on the
"always digest" flag on these news groups. This will instruct
nnmaster to treat all articles in the group as digests
(naturally, articles which are then found not to contain other articles
are still treated as normal articles.)
- C)ontrol
-
This is a special flag for the control group. It indicates that the
"Newsgroups:" field in the article header cannot be trusted (it
does not specify the groups to which the article has been posted.)
- D)irectory
missing
-
This flag indicates that the spool directory for the news group cannot be
found (the group has probably been removed with rmgroup(1M)). It is
set automatically be the nnmaster if it cannot access the
directory. When the flag is set, nnmaster completely ignores the
group, so it can be used to disable news collection in specific groups. If
you recreate the group or the directory manually, you must also clear this
flag to have the nnmaster recognize the group again.
- M)oderated
-
Indicates that the group is moderated. This flag is normally initialized
automatically from the active file, and it should not be changed
lightly.
- N)ever_digest
-
This is the opposite of the "always digest" flag; when set, the
nnmaster will never attempt to split any articles in that group
into subarticles.
When you select the (S)tat operation in the main or master menus,
you will get some general statistics about the database:
- initialized
-
The time when the database was last rebuild using nnmaster -I.
- last_scan,
last_size
-
The time stamp on the active file and its size the last time the
nnmaster read it.
- no of groups
-
The total number of groups in the database.
- Articles
-
The total number of articles in all groups. This is not an exact number,
because it will count split digests as a single article (making the number
too small), and it may count some articles that have been expired (making
the number too large).
- Disk usage
-
The total number of (1 kbyte) disk blocks occupied by the database.
The master index entries displayed when you select the (H)eader
operation in the master and group menus contain the following
information:
- group_name
group_number
-
The first line of the display will show the name of the group and the
internal group number which is used to identify the group in the
database.
- first/last
art
-
This is the numbers of the first and last article that are currently stored
in the database.
- active info
-
This is the numbers of the first and last article in the news system as read
from the active file. They will normally match the numbers above, but they
may differ while the nnmaster is working on the group (or it has
not yet collected all the articles in the group).
- Offsets:
index->..., data->...
-
These values show the starting position for the next write operation on the
index and data files. They are primarily used for consistency checking and
recovery after a system crash, but after an "expire by rewrite"
operation (expire method 2) which is performed "in-situ", the
data and index files may physically be longer than the actual data stored
in them.
- Flags:
-
This shows the current flags set for this group. If no flags are set, the
field is omitted from the display. One extra flag which was not explained
above is the BLOCKED flag; it is a temporary locking flag set on a group
by the nnmaster while it is updating the database files for that
group to prevent nn clients to access that group.
When you select the (D)ata operation on the group menu, you will
get a combined display of the raw data and index files for that group. The
index file contains a single 32 bit value for each existing article number.
This value is an offset into the data file pointing to the header for the
corresponding article.
When nn want to access the article from number N to the
last article, it looks up the offset for article number N in the index file,
and uses this as the starting point for reading article header information
in the data file. It then simply reads to the end of the data file in which
the article headers for articles number N+1, N+2, and so on follows
immediately after the header for article number N.
The article header information is presented in a very terse form;
each of the output lines are described below for reference purposes:
- offset = xxxx ,
article # = nnnnn (type)
-
This shows the offset into the data file and the article number. The offset
is stored in the index file for quick access. If no type is printed
it is a normal article. Other types are: "digest header" and
"digest sub-article".
- xpost(count):
nnn, nnn, nnn, ...
-
Cross-postings to other groups are encoded as a list of internal group
numbers.
- ts=nn hp=nn
fp=nn lp=nn ref=nn[+Re] lines=nn
-
These values are used by nn to sort, present, and access an article:
ts is the time stamp on the article; it is a simple encoding
of the posting date and time found in the Date: field.
hp, fp, and lp are offsets into the file containing the
article text: the header position, first text
position, and last text position. The first will be zero for
normal articles, but not for articles in a split digest. The last will be
equal to the length of the file for normal articles, but not inside
digests.
ref is the number of references on the Reference: line. If
"+Re" follows the number, the subject line contained a
"Re:" prefix which has been removed.
- Sender(length):
name
-
The name of the sender in "ready to print" format, i.e. reduced to
16 characters as explained in the nn manual.
- Subj(length):
subject
-
This is the full subject line from the article header (except for Re:
prefixes in various formats).
The $db, $lib, and $news used below are synonyms for the
DB_DIRECTORY, LIB_DIRECTORY, and the news system's lib directories
respectively.
$db/MASTER Database master index
$db/GROUPS News group names in MASTER file order
$db/DATA/nnn.x Index file for group number nnn
$db/DATA/nnn.d Data file for group number nnn
$master/GATE Message channel from nnadmin to nnmaster
$master/MPID The process id of the nnmaster daemon.
$Log The log file (truncate it regularly!)
The MASTER file contains a record for each news group, occurring
in the same sequence as the group names in the GROUPS file. The sequence
also defines the group numbers used to identify the files in the database
and in a few other places.
The GATE file will be created by nnadmin when needed, and
removed by nnmaster when it has read it. Therefore, to send a message
to the nnmaster requires that you are allowed to write in the $master
directory.
The GATE file is created with the owner and modes of the user that
runs nnadmin which may cause problems if the owner of the
nnmaster process (normally "news") is not allowed to read
the created GATE file (a "umask" of 022 is ok.) Unless you allow
ordinary users to create files in the LIB directory where the GATE file
resides, only the owner of the directory (normally "news") and
"root" can use nnadmin to send messages to the
nnmaster. However, to send a wakeup signal to the master, anybody can
run
nnmaster -w
The user interface is completely out of line with the rest of the
nn family, and the way to run nnadmin in the non-interactive
mode is a bit bizarre. This is not likely to change, because I believe there
are more important things to do!
Kim F. Storm, Texas Instruments A/S, Denmark
E-mail: storm@texas.dk