avenger(1) | Mail Avenger 0.8.5 | avenger(1) |
avenger - Mail Avenger
Mail Avenger is a highly-configurable MTA-independent SMTP (Simple Mail Transport Protocol) server designed to let you filter and fight SPAM before accepting incoming mail from a client machine. avenger is the script run on behalf of each user to decide whether to accept incoming mail.
When a client attempts to send mail to a user on the system, the avenger SMTP daemon, asmtpd, runs avenger to process the file .avenger/rcpt in the user's home directory. That file, a shell script with access to special functions, determines how the SMTP server should proceed. The possible outcomes are:
Mail Avenger should typically be configured to have a Separator character, allowing each user to maintain multiple email addresses. With sendmail, Separator is typically "+", with qmail it is typically "-". If the separator is "+", then any email sent to user+ext@your-host will be processed by files in user's .avenger directory.
Avenger first checks for a file named rcpt+ext in a user's .avenger directory, then for rcpt+default. If ext itself contains the separator character, for example user+ext1+ext2@your-host, avenger will check first for rcpt+ext1+ext2, then for rcpt+ext1+default, then for rcpt+default. The same algorithm is extended for arbitrarily many separator characters. (If separator is "-", simply replace "+" with "-" throughout the above description, including in the names of files such as rcpt-default.)
If mail is rejected by the recipient checks but the sender address of a message is local and UserMail is 1 in asmtpd.conf (which is not the default), then before rejecting mail, avenger will be run on behalf of the sending user. In this case, the address will be parsed as above, but avenger will look for rules in files beginning mail instead of rcpt. This mechanism can be used by local users who want to relay mail through the server from an untrusted IP address.
Using the mail configuration files, each user can, for instance, configure a mail+... file to accept mail from an IP address he or she trusts, even if that address is not trusted by all users. (Alternatively, using tools such as macutil, a user might set up relaying of mail in which the envelope sender contains a cryptographic code, checked by the mail+... script.)
Error output of an avenger script rcpt+ext or mail+ext is redirected to a file called log+ext in the same directory, for use in debugging.
Avenger configuration files are simply shell scripts, using the syntax described in sh(1). Each line of the file contains a variable assignment, command, or function to run. Scripts can additionally make use of a number of avenger-specific functions and variables. This section describes avenger functions. The next two sections describe variables.
Error output from command will be redirected to the same log file as output from the rcpt+... avenger script invoking the bodytest function. Standard output of command will be included as a diagnostic the bounce message if the exit code defers or rejects the mail.
Note that command and the arguments passed to bodytest will be run by the shell. Thus, it is important not to pass any arguments that might contain shell metacharacters such as ">" and "$".
The parameters can be tuned by setting variables in the script. The default values are:
greylist_delay=30m # Time to wait before allowing message greylist_ttl1=5h # How long to remember first-time senders greylist_ttl2=36D # How long to remember ok senders
m means minutes, h hours, and D days. For a complete list of allowed suffixes, see the documentation for dbutil(1) (in particular for the --expire option).
sender-key, if supplied, is used to identify the sender. The default value is "$CLIENT_IP $RECIPIENT $SENDER". If, for example, you wanted to record only the first 24-bits of IP address and didn't care about the recipient, you could use the command:
spf MYSPF +include:spf.trusted-forwarder.org \ +ptr:yahoo.com -exists:%{ir}.bl.spamcop.net ?all setvars case "$MYSPF" in pass) accept "I like you" ;; fail) reject "I don't like you" ;; error) # Note, could instead fall through to default here defer "Temporary DNS error" ;; esac
Note that commands spf0 and spf are synonymous, but spf is deprecated, because in a later release of Mail Avenger spf will become synonymous with spf1.
These variables are set by the avenger script. In addition, asmtpd sets a number of environment variables before running avenger. These are documented in the next section, ENVIRONMENT.
However, for special avenger files like unknown and default, it can contain useful information, because unlike the RECIPIENT_LOCAL environment variable, AVUSER reflects substitutions from the Mail Avenger domains and aliases files.
user is the user name for the connection reported by the client, if the client supports the RFC 1413 identification protocol, otherwise it is omitted. host is a verified DNS hostname for the IP, if asmtpd could find one. Otherwise, it is simply the numeric IP address.
Where the fields are as follows:
match -q "*Windows*" "$CLIENT_SYNOS" && greylist
list: user-mylist
Then avenger will be run on behalf of "user" (because alias expansion yields user-mylist-subscribe). EXT will be set to mylist-subscribe.
Note that EXT is empty when there is no suffix, and that it is equal to the name of the system file being processed when avenger is run on a system file. Like RECIPIENT, this variable is not set for bodytest commands.
openssl x509 -noout -issuer -in cert.pem
Note that this variable is mostly useful if the SSLCAcert file you have given to Mail Avenger contains more than one certificate authority, or signs other CA certificates. Mail Avenger will not accept client certificates if it does not recognize the signer of the certificate.
openssl x509 -noout -subject -in cert.pem
avenger is just a simple shell script. You can inspect the file to see what it is doing. Most of the interesting operations happen in either asmtpd, or in external programs spawned from avenger. This section documents the interface between asmtpd and avenger.
avenger inherits a unix-domain socket connected to asmtpd on its standard input and output. It sends commands to asmtpd over this socket, and similarly reads replies from it. In order to avoid mixing messages to and from asmtpd with the output of other programs you run, however, the avenger shell script reorganizes its file descriptors so that all communication to and from asmtpd happens over file descriptor number 3.
Each command consists of a single line, followed by a newline (except the return command, which can optionally take multiple lines). There may or may not be a reply, possibly depending on the outcome of the command. Most replies consist of zero or more lines of the form
VARIABLE is typically a variable name that was supplied as part of the command. The avenger shell script records results by setting the environment variable VARIABLE to value, so that it can be accessed by subsequent lines of the script.
Replies are sent in the order in which the corresponding commands were received. However, asmtpd executes requests asynchronously. Thus, one can perform several concurrent operations (such as DNS requests or SPF tests) by simply writing multiple commands to asmtpd before receiving any of the responses.
The "." command is a no-op, but asmtpd echoes the "." back to avenger as the reply. This allows one to synchronize the avenger process's state after issuing one or more commands. For example, one might issue several DNS lookups to check various RBLs (real-time blackhole lists), then issue a . command, then wait for replies. When the . comes back, all previous commands will also have completed. The avenger setvars command simply sends a ".", then loops until it reads back the ".", setting variables from any previous commands whose replies it reads in the process.
The following commands are available:
Note that asmtpd can only run one bodytest command per message. If there are multiple recipients of a message, all must run the same bodytest under the same user ID. If two users wish to run different bodytest commands, or even run the same command under different user IDs, asmtpd will defer the second SMTP "RCPT" command with the message:
This will cause the mail client to re-send the message later to the second user. To avoid forcing clients to send multiple copies of messages, you can place bodytest commands in system wide files (such as the default rule file), or use a redirect command to redirect to the AvengerUser, so that commands for multiple users can be run under the AvengerUser user ID.
Note that file descriptor 0 inherited by command is opened for both reading and writing. Thus, it is possible to modify the message before it is spooled by the local MTA. The command edinplace(1) is useful for running messages through spam filters that annotate messages before spooling them.
If no such A record exists, the reply is simply:
With the standard avenger script, this sets VARIABLE to the empty string. If there is a temporary error in DNS name resolution, there is no reply, and hence with the default avenger script VARIABLE will remain unset.
When checking such things as RBLs, it is advisable not to reject mail because of a temporary DNS error. You can use the shell construct ${VARIABLE-default}$ to return $VARIABLE when VARIABLE is set, and default when VARIABLE is not set. Similarly ${VARIABLE+set} returns set if VARIABLE is set, and the empty string otherwise.
For example, if bad-senders.org contained an RBL of undesirable sender hosts:
echo dns-a BADSENDER "$SENDER_HOST".bad-senders.org >&3 setvars test -n "$BADSENDER" && reject "$SENDER_HOST is a bad sender" test -z "${BADSENDER+set}" \ && defer "$SENDER_HOST.bad-senders.org: DNS error"
Note that when using the avenger script, there is already a function rbl to check RBLs.
Where priority-1 is the MX priority of host-1. As before, an empty string indicates no MX records exist, and no reply indicates an error.
#hops is the total number of network hops to IP-address if asmtpd can figure this out. (It won't always be able to if IP-address is behind a firewall.) If asmtpd cannot figure this out, the value is -1. hop1 and the remaining arguments are the addresses of routers along the way to IP-address.
local can be a local user name, or a local user name followed by the separator character and an extension. The name is mapped using the aliases (specified by AliasFile in asmtpd.conf).
Note that while the AvengerUser user can redirect to other users, ordinary users can only redirect to themselves or the AvengerUser.
The first form of this command (with a space between code and explanation) gives a single line explanation along with the result code. In the second form, avenger specifies a multi-line response. In this case all but the last line must contain a - between the code and explanation, while the last line must contain a space. (Note that the return keyword only appears on the first line; after starting to issue a return command, no further commands can be issued.)
where, for spf0, disposition is one of: none, neutral, pass, fail, softfail, error, or unknown (though the disposition none is actually impossible). For spf1, the equivalent disposition names are None, Neutral, Pass, Fail, SoftFail, TempError, PermError. (Currently spf is a synonym for spf0, but it is recommended that you avoid using spf as in a future release it may become an alias for spf1.)
As an example, suppose that your username is "joe", Separator is "+", and you have subscribed to a number of yahoo mailing lists using email address "joe+yahoo". If spammers started sending mail to "joe+yahoo", you would want to reject all mail to that address except that originating from yahoo's computers. Yahoo's computers might correspond to anything ending ".yahoo.com" or sharing a 24-bit IP-address prefix with any of yahoo.com's MX records. This can be accomplished with the following script in $HOME/.avenger/rcpt+yahoo:
echo spf YAHOO ptr:yahoo.com mx:yahoo.com/24 -all >&3 setvars case "$YAHOO" in fail) reject "Sorry, this private alias for Yahoo lists only" ;; error) defer "Sorry, temporary DNS error" ;; esac
If you never use your email address as an envelope sender, you can reject all bounces to that address with these commands in your rcpt file:
test -z "$SENDER" \ && reject "<$RECIPIENT> not a valid sender;" \ " should not receive bounces"
The following script runs spamassassin (a popular spam filter, available from <http://www.spamassassin.org/>) on the body of a message, unless the sender of the message has an SPF disposition of pass or is already going to be rejected by default.
# The next line immediately falls through to the default reject # disposition when mail has an SPF disposition of fail or the # sender does not accept bounce messages. errcheck test "$SPF" = pass \ || bodytest edinplace -x 111 spamassassin -e 100
The following script immediately accepts any mail from any machine at MIT or NYU (provided MAIL_ERROR is not set), "greylists" machines not in one of those domains, and if the greylist passes, falls through to the the default, system-wide rules:
errcheck spf TRUSTED ptr:nyu.edu ptr:mit.edu ?all setvars test pass = "$TRUSTED" && accept Trusted sender OK greylist_delay=5m greylist
The following script rejects mail from clients that have issued an SMTP "POST" command (which doesn't exist) or used aggressive, premature pipelining of commands. If the client put a space after the colon in the MAIL FROM: or RCPT TO: SMTP commands, it greylists the message using a key that includes the SYN fingerprint and first 24-bits of the IP address. If the SPF disposition of the message is error, it defers the message. If the SPF disposition of the message is softfail or none, it runs the body of the message through spamassassin.
errcheck test -n "$CLIENT_POST" -o -n "$CLIENT_PIPELINING" \ && reject "no spam please" test -n "$CLIENT_COLONSPACE" \ && greylist "${CLIENT_IP%.*} $CLIENT_SYNFP $SENDER" case "$SPF" in error) defer "Temporary error in SPF record processing" ;; softfail|none) bodytest edinplace -x 111 spamassassin -e 100 ;; esac
If you set your MACUTIL_SENDER environment variable to be "user+bounce+*@your.host.com" and send mail with macutil --sendmail, you can create the following rcpt+bounce+default to accept mail only to valid bounce addresses.
macutil --check "$SUFFIX" > /dev/null \ || reject "<$RECIPIENT>.. user unknown"
In conjunction with this script, you may want to reject bounce messages to your regular email addresss with your rcpt script, as described in the first example.
This example is slightly more complicated, and shows how to use a bodytest to reject mail based on message contents. The goal of this set-up is to check each message with the ClamAV anti-virus software (from <http://www.clamav.net/>) and the spamassassin mail filter. If the message contains a virus or is flagged as spam, it should be rejected with an explanation of the problem. We construct a shell script, $HOME/.avenger/body, to run these tests on message bodies. The script can be invoked with the line
in your $HOME/.avenger/rcpt file. Or, alternatively the script could be configured to run in the system-wide /etc/avenger/default file (in which case you want to make sure that the AvengerUser can write its own home directory, so as to store spamassassin files). The script is as follows:
#!/bin/sh out="`clamscan -i --no-summary --mbox - 2>&1`" if test "$?" = 1; then echo This message appears to be infected with a virus printf "%s\n" "$out" \ | sed -e '/Warning:/d' -e 's/^[^:]*: //' | sort -u exit 100 fi out="`edinplace -x 111 spamassassin -e 100`" case "$?" in 0) exit 0 ;; 100) echo Sorry, spamassassin has flagged your message as spam while read a b c; do test "$a $b" = "Content analysis" && break done read a read a read a while read a b c; do case "$a" in "") break ;; -*) ;; [0-9]*) printf " %s\n" "$c" ;; *) printf " %s\n" "$a $b $c" ;; esac done exit 100 ;; *) if test -n "$out"; then echo spamassassin failure: printf "%s\n" "$out" else echo system error in spamassassin fi exit 111 ;; esac
The first half of this script runs the clamscan virus checker, storing the output in variable out. clamscan exits with code 1 when a virus is found, exits 0 on success, and uses other error codes to indicate various system errors. We only want to reject mail if clamscan exits with code 1. When this happens, we take the output of clamscan, format it in a more pleasing way (stripping out warnings), and send it to standard output. An example of an SMTP transaction using this bodytest and detecting a virus will look like this (tested with the special EICAR test string that flags a positive with most virus checkers):
DATA 354 enter mail, end with "." on a line by itself Subject: eicar test X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H* . 554-This message appears to be infected with a virus 554 Eicar-Test-Signature FOUND
If the virus check fails, the script runs the message through spamassassin to check for spam. Note that spamassassin modifies the mail message, so that we must run it with edinplace. Note also that clamscan will read to the end of the input file, but this is okay since edinplace rewinds its standard input. We use the -e flag to tell spamassassin to exit 100 on spam. Then, if spamassassin exits 0, we accept the mail. If it exits with anything but 100, something went wrong and we temporarily defer the mail. Note that it might also be possible to accept the mail at this point, but since spamassassin edits the file in place, the message may be truncated if spamassassin exits unexpectedly.
If spamassassin exits 100, we reject the mail. We also report on why spamassassin has rejected the mail. Here again we take advantage of the fact that edinplace rewinds its standard input both before and after processing a message. Because the file descriptor has been rewound, we can start processing the message one line at a time with the shell script. Spamassassin by default (if you have not configred it with "report_safe 0") contains a spam report like this:
Content analysis details: (11.7 points, 5.0 required) pts rule name description ---- --------------- -------------------------------------------------- 1.0 RATWARE_RCVD_AT Bulk email fingerprint (Received @) found 4.2 X_MESSAGE_INFO Bulk email fingerprint (X-Message-Info) found 0.0 MONEY_BACK BODY: Money back guarantee 0.5 BIZ_TLD URI: Contains a URL in the BIZ top-level domain 0.6 URIBL_SBL Contains a URL listed in the SBL blocklist [URIs: crocpeptide.biz] 0.5 URIBL_WS_SURBL Contains a URL listed in the WS SURBL blocklist [URIs: crocpeptide.biz] ...
We skip over the headers, and for each result, print it to the SMTP session. Negative/whitelist results (those starting -), we do not report, and comment lines (not starting with a number) we print indented. A typical SMTP session looks like this (using the special GTUBE test line that triggers spam filters):
DATA 354 enter mail, end with "." on a line by itself Subject: gtube test XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X . 554-Sorry, spamassassin has flagged your message as spam 554- Missing Date: header 554 BODY: Generic Test for Unsolicited Bulk Email
Here's an example of how to use SSL client certificates for authentication. If you have a private CA with common name "My CA" that signs the certificates of all your authorized mail clients, you can place the following in /etc/avenger/relay to permit those clients to relay:
test "My CA" = "$SSL_ISSUER" \ && accept "Relaying permitted for client $SSL_SUBJECT" reject "relaying denied"
/usr/local/libexec/avenger, /etc/avenger/default, $HOME/.avenger/rcpt, $HOME/.avenger/rcpt* $HOME/.avenger/mail, $HOME/.avenger/mail*
dbutil(1), deliver(1), edinplace(1), escape(1), macutil(1), match(1), synos(1), asmtpd.conf(5), asmtpd(8), avenger.local(8)
The Mail Avenger home page: <http://www.mailavenger.org/>.
avenger (and the configuration files it reads) are shell scripts. In a shell script, it is sometimes tempting to use "echo ..." where one should instead use the command "printf '%s\n' ...". (The later just prints its argument to standard output, while the former interprets various "\" escape codes.)
In shell scripts, one must be careful about variables containing shell metacharacters. For example, it is not safe to run something like:
bodytest "echo $VAR > $PWD/log"
if variable "VAR" has untrusted contents that might contain characters like ">" or ";". The reason is that $VAR will be expanded and sent back to the SMTP server, which will then pass the expansion to the shell to execute the bodytest. ($VAR effectively gets expanded twice.) The escape utility can be used to avoid these problems. For example:
bodytest echo `escape "$VAR"` ">" $PWD/log
It is easy to forget to call setvars after a dns, rbl, or spf command.
David Mazieres
2018-10-09 | Mail Avenger 0.8.5 |