Using Real-Time Blackhole Lists For Filtering Email
Filtering email requires finding patterns that distinguish spam from business email. Patterns must match the greatest set of spam as defined by your email policy, while matching only the smallest set of business email. Messages incorrectly identified as spam are considered false positives. Alternatively, messages incorrectly identified as legitimate mail are considered false negatives.
This article describes the best way to filter spam using Real-time Blackhole Lists (RBLs).
Spam relay hosts are SMTP servers that either originate spam email or relay spam. A Real-Time Blackhole List (RBL) is a Domain Name Server (DNS) server that contains the IP addresses of SMTP servers considered to be spam relay hosts. RBL filtering operates on the IP address of the SMTP server sending an inbound email. An example spam header follows:
Microsoft Mail Internet Headers Version 2.0
Received: from ([127.0.0.2]) by with Microsoft SMTPSVC(5.0.2195.5329);
Sun, 2 Mar 2003 10:41:31 -0500
Subject: a new way to sto start the new years jfqbzsoe
Date: Sun, 02 Mar 03 10:18:57 GMT
X-Mailer: Microsoft Outlook, Build 10.0.2616
The IP address of the sender SMTP server is stated in the received field as "127.0.0.2". The IP is reversed and prepended to the RBL rule for a DNS look-up against the RBL DNS server. An example follows of the DNS look-up:
Email is considered spam if the DNS look-up resolves as an IP address. An example follows of a DNS look-up, conducted from a command prompt:
Addresses: 127.0.0.9, 127.0.0.2, 127.0.0.3, 127.0.0.4
Comparing Free and Subscription-Based RBLs
Free and subscription-based RBLs are available on the Web. Free RBL servers, such as http://relays.osirusoft.com/, http://www.spamcop.net and http://www.dsbl.org, are available to everyone at no cost. During times of high email traffic on the Internet, free RBL servers become very busy and will slow down or even fail to respond to DNS queries.
Subscription-based RBL servers, such as http://www.mail-abuse.org/, http:// www.postfixgate.com/, and http://www.maildeflector.net/ are available to subscribers. DNS transfers from the RBL server to the paying customer's local DNS server are available. RBL queries are then made to the local DNS server removing the dependance upon public RBL servers and internet bandwidth.
Each RBL has a policy detailing how an SMTP server qualifies as a spam relay host. Various RBL lists define spam relay hosts using different methods. Examples of RBL policies follows:
|Unconfirmed reports from spam recipients
|Confirmed reports from spam recipients
|Automated testing of SMTP server for open relay capabilities
||IP ranges of country's suspected of sending spam
|Spam trap reports generated by mailboxes which have never sent an outbound message
|Non-rfc compliant SMTP servers
Each RBL policy is unique - it relates to some organizations better than others. Policies based on unconfirmed reports and automated testing are considered to be aggressive. Large Internet Service Providers (ISPs) are often included on RBLs that have aggressive policies. Policies based soley on confirmed reports will block less spam; however, they will trigger fewer false positives during filtering. Ensure the policy matches your corporate policy of what is considered a spam relay host.
Utilizing the RBL features of speed, reduced bandwidth and low administration requires specific features as follows:
- RBL Weighting
- Asynchronous Queries
- Dropping the SMTP Session
- Excluding Friendly IPs
- Exchange Server Integration
Aggressive approaches, such as automated testing and countries suspected of sending, may give an unsatisfactory level of false positives. However, aggressive approaches do give clues to potential spam relay hosts. If enough of the clues occur, then it is desirable to filter a message as spam. A simple weighting procedure with three defined weight levels of "reliable", "potential", and "unconfirmed" is invaluable for reducing the number of false positives:
- A message triggering an RBL with a weight of "reliable" would always trigger
- A message triggering in 3 RBLs with a weight of "potential" would always trigger
- A message triggering in 5 RBLs with a weight of "unconfirmed" would always trigger
Asynchronous RBL Queries
Synchronous queries occur sequentually, increasing the filtering for each email time to the total of each DNS look-up response. Asynchronous RBL queries occur simultaneously, thereby reducing the filtering time for each email. Email from spam relays will be filtered on the fastest RBL result, indicating a spam relay. Email from legitimate SMTP servers will be filtered on the slowest DNS look-up response.
Dropping the SMTP Session
RBL filtering operates on the IP address of the server, initiating an SMTP session with the recipient SMTP server. The server initiating the session is considered a spam relay if a RBL DNS look-up resolves to an IP address. Once the server is considered a spam relay, the SMTP session can be dropped before the message header and body are downloaded. Spammers address email to domains with popular aliases in an attempt to find valid users. This method, known as "dictionary attacks", propagate spam and find valid addresses that do not return Non Delivery Reports (NDR). During dictionary attacks, dropping the communication saves bandwidth as other methods of filtering (such as address or message content filtering) require the message header and body to be downloaded.
Excluding Friendly IPs
Legitimate customers may have unsecured SMTP servers that are relaying spam from a third party. Notify the customer's email administrator and add the problem server as a temporary Friendly IP. IP addresses listed as friendly will bypass the RBL filter. For customers with a group of unsecured SMTP servers, then it is desirable to specify the Friendly IP with a netmask. Spammers typically falsify their originating SMTP address, so basing exceptions upon the originating SMTP address is a potential security risk.
www.openrbl.org is an excellent RBL look-up resource for IPs or hostnames. Information is available about the registered hostname and netblock, as well as which RBLs are listed for a particular IP or hostname.
Inbound messages that are considered spam and are not caught by the RBL filtering should be reported to the RBL sites. Each RBL site has forms available to nominate IP addresses for SPAM status.
Business email caught by RBL filtering should be reported to the SMTP server's administrator. The administrator may be unaware that he or she is listed on a RBL and is forwarding spam to third parties by means of open relay.
Exchange Server Integration
Many secure content management tools require an additional proxy server or gateway to work. This can translate into time-consuming modifications, or even reconfiguration to Exchange or its connectors. By selecting a software tool that can be "bolted-on" directly to the SMTP/Internet Connector and MTA, all configuration and operation can be performed as a straightforward add-in to the Exchange server, through Exchange's native facilities, services and connectors.
Spam filtering then becomes synchronous with the message flow, honoring your security settings, enabling seamless, uncompromised support in a single application, and improving performance, reliability and ease of administration.
About the Author and Nemx Software
John Young, President of Nemx Software, has more than 20 years of experience in software development. Nemx's family of secure content management and antivirus products for the Exchange Server environment offers all the features and capabilities of a Fortune 500 solution - all mentioned in the above checklist - tailored to the specific needs of SMBs. Nemx's Power Tools for Microsoft Exchange 2000 is the first native Exchange product to employ real-time Reverse Blacklist Technology (RBL), enabling users to effectively "kill" 85% of incoming spam, increase overall Exchange performance, decrease message storage requirements, and provide efficient control of inaccurate blacklist databases.