email system

master.cf

Define the mfilter service:

smtp      inet  n       -       -       -       -       smtpd
pickup    fifo  n       -       -       60      1       pickup
cleanup   unix  n       -       -       -       0       cleanup
qmgr      fifo  n       -       -       300     1       qmgr
rewrite   unix  -       -       -       -       -       trivial-rewrite
bounce    unix  -       -       -       -       0       bounce
defer     unix  -       -       -       -       0       bounce
trace     unix  -       -       -       -       0       bounce
verify    unix  -       -       -       -       1       verify
flush     unix  n       -       -       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
smtp      unix  -       -       -       -       -       smtp
relay     unix  -       -       -       -       -       smtp
showq     unix  n       -       -       -       -       showq
error     unix  -       -       -       -       -       error
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil

mfilter   unix  -       n       n       -       -       pipe
  flags=Rq user=cyrus argv=/var/spool/cyrus/bin/mfilter_transport ${user} ${extension}

Spam folder

Whenever you create a folder for a user, make sure to also make a spam folder for him (I'm not sure what happens in Debian if cyrdeliver is called with an unexisting folder; will it create it? If so, this entire subsection should be removed).

Training

Root's ~/.fetchmailrc:

poll localhost
  protocol imap
  user cyrus
  password PWPWPWPW
  fetchall
  no rewrite

Avoid encryption since we're connecting to localhost. :-)

~cyrus/bin/sa-learn-pending:

for user in $(getent passwd | cut -f1 -d:); do
  if groups $user | grep mailusers >/dev/null; then
    for type in ham spam; do
      fetchmail --folder user.$user.learn-as-$type --silent --mda "( echo -n 'From %F ' ; date '+%a %b %e %R:%S %Y' ; /bin/cat ; echo ) >> /tmp/mbox.$$";
      if test -f /tmp/mbox.$$; then
        su $user -c "sa-learn --$type --mbox" </tmp/mbox.$$;
        rm /tmp/mbox.$$;
      fi
    done
  fi
done

If you figure out how to suppress the fetchmail: Server CommonName mismatch: freaks-unidos.net != localhost errors (that still only receives unencrypted imap connections through the loopback interface), let me know.

Make a crontab for user root that runs this script. I choose to run it once every day, but you can pick a different interval.

11 2 * * * ~cyrus/bin/sa-learn-pending 2>/dev/null >/dev/null

Here are a few notes:

• The crontab dumps the output of the sa-learn-pending command. We shouldn't do that. But then, the CommonName mismatch error would make things very verbose (and I don't want to just grep it out, that adds more complexity).
• The sa-learn-pending scripts does this:
  • For every user in the mailusers group:
    • Fetch all learn-as-ham into a mbox folder using fetchmail and then learn all of them. Do the same for the learn-as-spam messages.
• The --mda parameter in sa-learn-pending isn't very reliable; it would be best to use a program to deliver messages to the mbox with better error reporting. If you figure out one to use instead, let me know. I'll use it for the time being since losing messages there (before the training) isn't that bad (we're erasing them anyway!).

Configuration

cm cyrus.spam
cm cyrus.virus
dam cyrus.spam anyone
dam cyrus.virus anyone
sam cyrus.spam cyrus all
sam cyrus.virus cyrus all

Filter script

~cyrus/bin/mfilter_transport, what a sexy script:

#!/bin/sh

USER=$1
MAILBOX=$2

EX_OK=0
EX_TEMPFAIL=75
EX_UNAVAILABLE=69

VIRUS_RETURN=$EX_OK;

/usr/bin/spamc -u $USER >/tmp/filter.$$ || { echo "Unable to save mail: /tmp/filter.$$"; rm -f /tmp/filter.$$; exit $EX_TEMPFAIL; }

CLAMAV=$(/usr/bin/clamdscan --disable-summary --stdout - </tmp/filter.$$)

if echo $CLAMAV | grep -q FOUND; then
  MAILBOX=virus;
  USER=cyrus;
  FORCE_EXIT=$EX_OK;
elif echo $CLAMAV | grep -q ERROR; then
  echo $CLAMAV;
  rm -f /tmp/filter.$$;
  exit $EX_TEMPFAIL;
elif egrep -q "^X-Spam-Level: \*{12,}" /tmp/filter.$$; then
  MAILBOX=spam;
  USER=cyrus;
  FORCE_EXIT=$EX_UNAVAILABLE;
elif grep -q "^X-Spam-Flag: YES" /tmp/filter.$$; then
  MAILBOX=spam;
fi;

/usr/sbin/cyrdeliver -a "$USER" -e -m "$MAILBOX" "$USER" </tmp/filter.$$
RET=$?;
rm -f /tmp/filter.$$;
test $FORCE_EXIT && exit $FORCE_EXIT
exit $RET;

Creating a user

Suppose we're creating user azul.

cm user.azul
cm user.azul.spam
cm user.azul.learn-as-spam
cm user.azul.learn-as-ham
sam user.azul.learn-as-spam cyrus lrds
sam user.azul.learn-as-ham cyrus lrds

$ razor-client
$ razor-admin --create
$ razor-admin --register

Filtering mails

Right now setting up filter is rather difficult: it assumes knowledge of Sieve as well as the capability of login into the machine (eg. using an SSH client or some other similar method) and running commands on it.

require ["fileinto"];

if header :contains "list-post" "<mailto:users@subversion.tigris.org>"
{
  fileinto "INBOX.subversion-users";
  stop;
}

sieveshell localhost

put lists.script
activate lists
quit

Mutt

Mutt is my favorite email client. :-)

set folder=imaps://azul@freaks-unidos.net/INBOX;
set spoolfile==;
set postponed==Drafts;
set record==Sent;

#!/bin/sh

# I use two sed processes voluntarily (to easily handle both "From:
# foo@bar.com" as well as "From: foo <foo@bar.com>").

ADDRESS=$(grep ^From: | head -1 | sed 's/^From: *//' | sed 's/^.*<\(.*\)>.*$/\1/')

# Add it unless it is already there.

grep $ADDRESS ~/.mutt.scores.rc || echo "score '~f $ADDRESS' 10" >>~/.mutt.scores.rc

source "/home/azul/mutt.scores.rc";
color index brightwhite default '~n 10-';

macro index S "s=learn-as-spam\n" "learn message as spam and delete it";
macro index H "C=learn-as-ham\nsimaps://azul@freaks-unidos.net/INBOX.read\n|~/bin/mutt-from-score >/dev/null\n" "learn message as ham and move it to the read folder";