Antispam Tools: Can They Keep Up?

The deluge of spam just keeps increasing, and spammers are getting sneakier every day. That means antispam products have to get tougher to stay ahead of the game. So we tested the latest versions of five spam blockers to see if they can hold back the flood.

We configured each product according to the vendor's advice for minimizing the amount of legitimate mail misfiled in the spam folder (false positives) while still keeping the most spam out of our in-box (false negatives). Where there was a choice, we tested with Microsoft Outlook or Outlook Express, based on the vendor's recommendation.

Four of the five products we reviewed use filtering techniques—analyzing message content and headers—to identify spam. To test those, we diverted a stream of spam-infested, real-world messages to multiple e-mail accounts, each protected by one of the products. For comparison, we ran the same e-mail stream through Outlook's own spam filter and through Cloudmark SpamNet and Aladdin's Spam Catcher, both of which did well in our last roundup. We intended to compare Norton AntiSpam 2004, our Editors' Choice in that roundup, as well, but a glitch in the product (which has since been fixed) during our testing period prevented the necessary unattended mail processing.

For ten days, we manually trained the products, correcting any legitimate mail that was misfiled as spam or vice versa. Then we cleared the in-boxes and configured the programs to use only filtering to identify spam—not whitelists, blacklists, or challenge/response systems. We let them run for ten more days and then exported each product's mailbox using askSam 6. We identified the messages that were received by all of the products and discarded the rest. Next, we manually identified the remaining messages as undeniably spam or undeniably valid, deleting all others. The results, shown in the accompanying table, reflect only the product's ability to filter spam. When whitelisting, blacklisting, and challenge/response (if available) are enabled, accuracy will naturally improve.