Fosdem TODO Linux
2004 Edition Free and Open Source Software Developer's European Meeting


2004-01-19 - Tom Kistner

Anti-spam techniques

An interview conducted by Alain Buret
FOSDEM - First and traditional question : Please present yourself ...

Tom Kistner - I am 31, and currently a freelance Programmer and System Integrator/Administrator. My main focus is SMTP servers, mainly for relaying purposes (frontend to groupware servers like Exchange and Notes).

FOSDEM - How and why did you start looking at MTAs ?

Tom Kistner - Much like everyone else, I had my first SMTP experiences with sendmail. When I was set to the task of creating a flexible mail relay (with content scanning), I looked for alternatives and ended up using Exim 3.xx, with a short deviation via Qmail.

For the content scanning part (back then, this was just virus scanning), I was looking at AMaViS, whose integration was via the classic "pipe-and-resubmit" loop. This had the severe disadvantage of losing envelope addresses. I decided to reinvent the wheel and wrote the first incarnation of Exiscan that tied into Exim 3.xx's queueing mechanism. It was a perl script and like all perl scripts, it was rather ugly, but it gained a large following over time. Debian still distributes it in their stable distro ...

When Exim 4 was released, I ported the scanner "glue" to C, to be patched into Exim directly. This also allowed for rejects in the SMTP phase, which was a novelty when I first released it.

FOSDEM - Spam is a bad thing not only because it fills in our mailbox, but also because it's flooding the Net; do you think there is something to do on the sender side to stop it ? Make it illegal ? Would need some sort of "world law" ...

Tom Kistner - Since legal people usually have deficits in technical knowlegde, I doubt that laws will do much good. The first anti-spam law in the US (which went into effect this year) was actually applauded by spammers since it puts their "business" on legal ground. Those spammers who can't or don't want to comply with the laws will simply offshore their business to spam-friendly countries. Since spamming is business some ISP will always sell traffic to spam gangs, no matter if in China or in the US.

And about a "world law", I can't see this happen in the next millenium :)

FOSDEM - On the receiver side, there are some software available to filter mails. Can you explain the way these tools are working ?

Tom Kistner - There are many tools available for spam filtering, and they have one thing in common: They all suck. Sure, some do suck less (like RBLs and SpamAssassin), but others are only adding to the problem (like Challenge-Response systems).

Here are the basic types of anti-spam systems:

  • Host-based RBLs: The oldest method, and still the most reliable, if you choose the right lists.
  • Message analysis: Either SpamAssassin "classic" or Bayesian, or a combination of both.
  • Fuzzy Fingerprinting: Example: Vipul's Razor
  • Challenge-Reponse Systems: Example: TMDA.
  • Adding "Allowed sending hosts" to DNS: Example: SPF,RMX,DMP (single proposal seems to be well underway).

    In my talk, I'll show why all of them suck in at least one way ...

    FOSDEM - Regarding these softwares, what's the missing feature that you would like to see present?

    Tom Kistner - Without fundamentally changing the SMTP protocol, or everyone participating in schemes like SPF, there can be no satisfactory solution to the problem. As for features, I think that every even remotely possible solution has been tried ... the trick is to choose the right mix of solutions for your personal spam problem.

    FOSDEM - Introduce in few words what you're going to talk about during your presentation ...

    Tom Kistner - I'll split it into two parts. There'll be an overview about the current spam situation, with a (short) presentation of above mentioned anti-spam methods and their respective drawbacks. Then I'll go into technical details, showing some recipes for doing content scanning with Exim and the exiscan-acl patch.

    FOSDEM - What are you expecting from your talk at FOSDEM and from the interactions with other developers present at the event ?

    Tom Kistner - Well, it's my first time at FOSDEM, I'm looking forward to it, and I hope there'll be some relaxed interaction involving Belgian Beer. :)


  • © FOSDEM 2003-2004 - powered by Argon7