| SORTMAIL(1) | General Commands Manual | SORTMAIL(1) |
sortmail - classify incoming mail
sortmail [ -v ] [ -terse ] [ -home path ] [ -mailbox path ] [ -mailrc initfile ] [ -sortmailrc initfile ] [ -inbox filename ] [ -mbox ] [ -pop|pop3|pop2 user:password@host ] [ -pop|pop3|pop2 /path ] [ -keep ] [ -noapop ] [ var=value ] [ -dumpCrcs dbmname ] [ -verify ] [ -version ] username
Sortmail reads and classifies email according to patterns you specify. It can be used to process incoming mail, filter mailing lists, process mail folders or download mail from POP servers.
For processing incoming mail, create this .forward file in your home directory:
"| /path/sortmail user"
Where "/path/sortmail" is the full path where you installed sortmail, and user is your own userid. The userid must be specified because when mail arrives, sortmail could be run as root, daemon, or any number of other uid's.
Once your .forward file is set up, sortmail will classify incoming mail according to the patterns in $HOME/.sortmailrc. Your .sortmailrc file is similar to a news KILL file, but somewhat more powerful. You can discard mail, have it delivered to your mailbox, have it filed into a folder, forward it to another address or even pipe it through a shell command.
Don't let the long list of options and command below frighten you. New users might do well to skip to the EXAMPLES section below, and/or read the README and sample files in /usr/share/doc/sortmail.
When sortmail starts up, it first reads the following config files: /usr/lib/sortmailrc, /usr/local/lib/sortmailrc, /etc/sortmailrc, /usr/etc/sortmailrc, /usr/local/etc/sortmailrc, $HOME/.mailrc and $HOME/.sortmailrc.
Your .sortmailrc file is a series of lines in the form
set variable=value
/regular-expression/modifiers:command[:command...]
[ip-address]modifiers:commands..
[ip-address/mask]modifiers:commands..
[ip-address - ip-address]modifiers:commands..
(logical-expression):command[:command...]
includerc filename
listinclude filename
listexclude filename
header headerline
replace headerline
bouncecheck dbmname
where regular-expression is any ed(1)-style regular expression, modifier is any of i, t, f, s, h, a, o, and command is one of m, j, v, f file, a file, d file, +file, or | command. Multiple commands may be placed on a line, separated by ':'s. If you need to place a ':' within a command for any reason, escape it with '\'.
Users of rn-style KILLfiles will be familiar with this format.
The [ip-address] form specifies a literal IP address to be matched (e.g. 192.168.3.4) or a partial IP address (e.g. 192.168.3). This differs from a regular expression in that the '.' character must match literally, and the pattern must match at the start (e.g. 129.192.168.3 would not match the pattern given above.) (Note that the '[]' characters are literal here, and do not denote an optional argument.)
IP addresses may also be specified as a range, e.g. "[192.168.0.0/16]" would match all IP addresses containing 192.168 in the first 16 bits. Finally, IP addresses may also be specified as e.g. "[192.168.0.0 - "192.168.255.255]".
These modifiers affect how the regular expression is applied to the incoming mail. The default is 's'.
Any combination of s,t,f,h,a may be used. If none are specified, 's' is assumed.
These commands are executed for any message which matches a search pattern.
The following is a list of commands which may be contained within a .mailrc or .sortmailrc file.
User Name <address>
will only use the address part of the line. This allows the mailing list itself to be used as the include list.
If the value field is empty, the specified header line is deleted.
Variables are used in the form $name or ${name}. Variables may appear anywhere in the init file.
Sortmail uses the following variables, which may be changed in your .mailrc or .sortmailrc files. Variables may also be set on the command line.
In addition, sortmail defines the following environment variables before passing a message to another program.
Here is a sample .sortmailrc file:
set default=+other /MAILER-DAEMON/f:+bounces /falk/t:m /bldg8/t:m /joe/f:m:c /for brenda/s:k /scubaclub/t:+scuba /scuba/s:+scuba /marko/f:j /testing/t:m falk@lab /jym@apple/f:| /home/falk/bin/fixjim /^Precedence: junk/h:+other (/bill/f && /dive/s):+scuba [211.114.0.0/16]r:j
In this example, the folder directory and other variables have whatever values were specified in .mailrc. Unclassifiable mail will be sent to the folder "+other". Mail from "MAILER-DAEMON" is sent to the folder "+bounces". Mail to "falk" or "bldg8" is sent directly to my mailbox.
Mail from my friend joe is sent directly to my mailbox, and processing continues in order to see if there's somewhere else it should go as well.
Mail labeled "for brenda" is left at the POP server untouched. (This only works if the email is being downloaded from a POP server; it would be lost otherwise.)
Mail to the scuba club or with "scuba" in the subject line is sent to the "+scuba" folder. Mail from marko is thrown away unread. Mail to the "testing" alias is forwarded to my account on another machine.
Mail from my friend jym, who formats his mail in a funny way is passed through a shell script which cleans up his messages and appends them to my mailbox. Mail messages with "^Precedence: junk" anywhere in the header are filed in +other.
The next-to-last line shows a feature new to sortmail version 2: logical expressions. In this case, mail from bill with the subject "dive" is added to the scuba folder. Logical expressions are described in detail below.
Finally, the last line shows another feature new to sortmail version 2: IP ranges. In this case, all email with an IP address in the "211.114.0.0/16" range in a "Received:" line will be junked unread.
Note that the patterns are applied in the order given; it is important, for example, that the "MAILER-DAEMON" pattern precede the "falk" pattern so that mail from MAILER-DAEMON is filed in +bounces even if directed to me personally. Similarly, mail from marko will not be junked if addressed to me personally.
Logical expressions allow you to specify more complicated rules for processing mail. For example, you could specify that all mail from a certain domain with a size greater than a certain amount be deleted unread unless a specific keyword were to be found in the header.
Logical expressions consist of the following operators, grouped in order of precedence:
| n | integer constant |
| $var | variable. |
| /pattern/ | regular expression. Evaluates as 0 or 1. |
| ! | logical not |
| * | multiply |
| / | divide |
| + | add |
| - | subtract |
| < | less than |
| <= | less than or equal |
| > | greater than |
| >= | greater than or equal |
| == | equal |
| != | not equal |
| & | logical AND |
| && | logical AND |
| | | logical OR |
| || | logical OR |
| , | comma |
Order of precedence in expression evaluation may be modified by use of parenthesis.
´:' commands may follow any close-parenthesis or regular expression. See examples below.
The second form of logical AND and OR operations ("&&" and "||") are optimized in this way: If the left half of an AND is false, or the left half of an OR is true, then the right hand is not evaluated. Thus, you should place a simple expression (such as a subject match) to the left and a complex expression (such as a message body search) to the right. If the simple expression evalutes to false or true respectively, the complex expression is not tested.
The first form of logical AND and OR operations ("&" and "|") always test both sides of the expression.
The comma operator deserves a bit of explanation for those not familiar with the C language. The comma operator evalutes the expressions on both sides and returns the expression on the right -- ignoring the one on the left. Thus, the expression "3 , 4" evaluates as 4. The comma operator is useful only when the expression on the left has some sort of side effect when evaluated -- i.e. it contains ':' commands.
Here are some sample expressions:
mail from joe comes directly to me. This is the same as /joe/f:m
mail from joe with "dive" in the subject line goes to the scuba folder.
mail from joe without "dive" in the subject line goes directly to me. Else, mail from joe goes to the scuba folder.
Looks like joe posted another long boring vacation report to the scuba list. Junk it.
(/earthlink/r && $size > 32768 && !(/key west/ia || /caymans/ia) ):j
Junk it if it came from or passed through earthlink (as shown by the Received: lines), and the size is greater than 32k and it does not contain the phrase "key west" or "caymans" anywhere in the message body. Case is ignored in the body search. Note that we examine the message body last to avoid downloading the message unnecessarily.
Note also that logical expressions may be continued across multiple lines as needed.
An extremely simple expression. (1) is always true, so all mail that reaches this expression is filed to the folder "maillog". The ":c" command causes processing to continue.
This expression is a very good one to have at the top of your .sortmailrc when testing a new configuration. All incoming mail is copied to a backup log before more complex expressions are tested.
This example shows the use of ':' commands within an expression. Mail from joe goes to the "joemail" folder. If it also contains the subject "scuba", it goes to the scuba folder as well.
This example shows the use of the ',' operator. Mail from joe goes to the joemail folder. Whether or not this matches, the mail is tested again to see if it belongs in the scuba folder. If so, then processing is finished.
(/sex/:+sex && /drugs/:+drugs && /rock-n-roll/:+rock):+bacchanalia
This pattern does not do what it looks like it was intended to do. That is, at first glance it looks as if the pattern is intended to place all messages containing "sex" in the subject into the sex folder, all "drugs" messages into the drugs folder, all "rock-n-roll" messages into the rock folder and place messages into the bacchanalia folder if they match on all three keys. However, logical expressions are only evaluated as far as necessary. If the "sex" pattern is not matched, the next two will not be tested at all. A "rock-n-roll" message would be missed by this pattern.
In this case, the '&' operator should be used instead of '&&'.
Skip this section unless you're using sortmail to administer a mailing list.
In a homogeneous environment, it is usually not necessary to use sortmail or any other mail filter. You would simply create the alias in /etc/aliases and let sendmail(8) handle everything.
However, in a heterogeneous environment, there can be problems. The internet document Rfc822 specifies the handling of internet mail, but there are many mailers out there which do not honor Rfc822 and cause trouble. Not surprisingly, many of the major service providers are among the worst troublemakers.
What typically happens is that for some reason, some member of your mailing list suddenly cannot receive mail. The service provider at the user's end bounces an error message back to the list itself rather than to the original sender or the administrator. The error message is then resent to the list subscribers -- including the the one who cannot receive mail, causing another bounce. This creates a loop, sending and resending bounce messages to everybody on the loop every few minutes. Murphy's Law states that this will happen while you are on vacation.
Here is how to administer a mailing list:
First, (as root) edit /etc/aliases and add the following lines:
scubaclub: "| /usr/yourname/sortmail -sortmailrc scubaclubrc
yourname"
scubaclub-real: :include:/usr/yourname/scubalist
scubaclub-request: yourname
owner-scubaclub: yourname
The first entry indicates that mail to the scuba club goes through sortmail, using a specific sortmailrc file. The second entry is the actual scuba club alias to which sortmail will forward the mail. The third entry is a standard list address which will be used by users to contact you directly; this should always exist for any mailing list. The final entry is used by the sendmail system to send internal errors back to you.
(Most unix systems require you to run newaliases(8) after editing /etc/aliases.)
Second, create /usr/yourname/scubalist, containing the names and
addresses of everybody in the list.
yourname <youraddress>
Joe Shmoe <joe@foo.com>
Jane Doe <jane@bar.com>
(etc.)
Third, create a sortmailrc file which will be used to filter incoming mail.
# general variables set alias = scubaclub set owner = yourname set site = yourhost.com set digestDir = ~/Maillists/Scubaclub # mail that makes it through the filter gets mailed to # the list and archived. set default = m $alias-real@$site:a $digestDir/archive # mail that gets rejected is mailed to me set reject = m $owner # catch anything that looks like a bounce or a loop /Mailer-Daemon/f:m $owner /MAILER-DAEMON/f:m $owner /Postmaster/f:m $owner /scubaclub/f:m $owner /X-List-Name: scubaclub/h:m $owner bouncecheck $digestDir/bounceDb # (For some reason, we can't set Errors-To to $owner@$site, # because if we do, sendmail will expand $owner into an # invalid value before connecting to SMTP. It would probably # be ok if I didn't have a personal .forward file. By adding # a '\' to the address, we avoid the problem.) replace Reply-To: $alias@$site replace Errors-To: \\$owner@$site header Comment: send add/delete requests to $alias-request@$site header X-List-Name: $alias
In this example, the variables $alias, $owner, $site and $digestDir are not used internally by sortmail, but are created for convenience and generality.
The search patterns are used to detect possible mail loops, and as such, always send mail to the owner. Mail that makes it without matching any of the patterns is sent to the list.
As a last resort, the mail is processed by the bouncecheck command which maintains a database of previously-seen messages and will reject any message that seems to be a repeat.
If mail passes through all the patterns unmatched, it is probably a valid message. In this case, the message is processed by the commands in $other, which mail the message to the actual alias, and append a digest version of the message to ~/Maillists/Scubaclub/archive.
Finally, the headers of outgoing mail are modified. The "Reply-To:" header is added so that replies to mail from the list are sent to the list at large, and not just to the sender of the original message. The "Errors-To:" header is added so that bounces will be sent to the administrator instead of to the list in general. (Not all mail transfer agents honor the "Errors-To:" header.)
The "X-List-Name:" header line serves two purposes. First, it lets recipients know what they're receiving. Secondly, it is a trick used to help detect bounces. It is added so that it may be searched for in incoming mail. If an incoming message contains this header, it is likely that this is a bounce, and is sent to the administrator for inspection.
If some of your list members wish to receive messages in "digest" form, you can split the list into two sections, one normal and one for the members who want digests. Add the following line to /etc/aliases:
scubaclub-digest: :include:/usr/yourname/scubadigest
and change $default in /usr/yourname/scubaclubrc:
set default = m $alias-real@foo:a $digestDir/archive:d $digestDir/digest
Now, incoming messages will be copied to ~/Maillists/Scubaclub/digest as well as to ~/Maillists/Scubaclub/archive. On a nightly basis, execute a program that will test to see if ~/Maillists/Scubaclub/digest is non-empty, and if so, mail it to scubaclub-digest@yourhost and empty it.
#! /bin/sh # # collect the digest file, prepend some header info and transmit digest=/usr/yourname/Maillists/Scubaclub/digest alias=scubaclub host=yourhost.com if [ ! -s $digest ] ; then
exit 0 fi cat $digest | awk " BEGIN {
print \"Return-Path: $alias@$host\"
print \"Date: `date`\"
print \"From: $alias@$host\"
print \"To: $alias@$host\"
print \"Subject: $alias digest\"
print \"X-List-Name: $alias\"
print \"\"
print \"\" } {print}" | \ /usr/lib/sendmail -om -oi -f$alias@$host $alias-digest rm $digest touch $digest chmod a+w $digest
Remember that sortmail can be executed under any userid (e.g. root, daemon or the sender of the mail) depending on who sent the mail, and whether or not it came from the local machine. Because of this, you cannot depend on any user environment to be available, especially environment variables and path. All filenames and program names should be specified as full paths, except that "~", "~user" and "+folder" forms are understood. The permissions of sortmail and every directory along its path should be such that any user can execute it.
If you pipe incoming mail through a program, that program should not generate any output to stdout or stderr whatsoever. If it does, that output will be sent back to the originator of the mail as if the mail had bounced.
Always test your setup thoroughly, especially when administering mail lists. Mistakes usually result in bounce messages being sent to the originator of mail. This can be catastrophic with a mail list. When testing a mail list, start with a test list containing only your name and a known bad address to test bounce handling. Use of a logfile and -v is recommended for the first few days after installing.
The bounce detection mechanism tries to be robust, but as the saying goes, you can never make a system that's foolproof because some fools are ingenious. No matter how thorough the detection mechanism is, there is a broken mailer out there somewhere that can defeat it.
Never leave a mailing list unattended; that's when bounce loops always seem to start. If you go on vacation, either temporarily shut the list down, or designate someone who can turn it off in an emergency.
The following exit values are returned:
If multiple instances of sortmail are executing and printing status messages to the same logfile, the output may become jumbled.
The maxlines variable is not yet implemented.
Copyright 1990, 1999 by Edward A. Falk (falk@efalk.org)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
regex.[ch] is covered by the GNU copyleft.
| 21 Apr 1991 |