MailMarshal SMTP Anti-Spam Configuration

Transcription

1 Technical White Paper MailMarshal SMTP Anti-Spam Configuration Contents Best Practices 2 Optimum Base Configuration 2 SpamProfiler 3 SpamCensor 4 SpamBotCensor 6 URLCensor 6 Reputation Services (DNS Blacklists) 9 CountryCensor 14 HELO Rules 17 TextCensor Scripts 18 Attack Prevention 19 Spam Management Tips 19 Whitelisting Practices 22 Submitting Missed Spam Or False Positives to M86 Security 26 Common Pitfalls 27 Conclusion 28 MailMarshal SMTP provides an excellent spam detection rate out of the box. Basic configuration settings are covered in the white paper MailMarshal SMTP Anti-Spam Basics. You should read that paper and review your MailMarshal SMTP configuration before continuing to this paper. This paper provides detailed information about anti-spam functions in MailMarshal SMTP and explains how to adjust the settings of anti-spam options including SpamCensor, SpamProfiler, CountryCensor, and other items. The information presented has been updated for MailMarshal SMTP 6.7. This paper requires an intermediate level technical understanding of concepts and MailMarshal configuration. To fully understand and apply the ideas in this paper, you should be familiar with the MailMarshal Configurator, the registry editor, and text configuration files. m86security.com

2 MailMarshal SMTP continues the tradition of providing a comprehensive tool to control spam based on an extensive array of functionality. The two key concepts are detection and management. MailMarshal SMTP uses technologies that enable high spam detection rates and few false positives, with easy administration and a variety of precise customization options. It does this within the context of an integrated content management package. MailMarshal SMTP is more than an anti-spam system it provides organizations with the means to control all content, including spam, viruses, text, and attachments, within a rules-based framework. BEST PRACTICES Whether a system has been installed cleanly with all of the default rules in place, or upgraded from a number of major versions back, it is very important to ensure that MailMarshal SMTP is taking advantage of all of the features available. Spam updates are retrieved automatically through the web, but M86 Security will not make modifications or additions to existing rules. In order to ensure that you are using the latest technology released, and to make use of the files downloaded through the automatic updates, a base configuration should normally have a number of anti- Spam features enabled. These include: SpamCensor SpamBotCensor SpamProfiler Reputation Services (DNS blacklists) A number of other functions available within MailMarshal SMTP can also be used for anti-spam purposes. These include: URLCensor, to check for links to domains frequently advertised in spam URLCensorIP, to check for links that resolve to IP addresses frequently used in spam CountryCensor, to check the country of origin of messages HELO rules, which are used to examine the behavior of the connecting SMTP system Attack Prevention capabilities, which allow you to enforce network-friendly behavior on the part of connecting systems TextCensor scripts, which provide you with a simple way of updating spam detection capabilities on the fly before automatic spam updates are released. The following sections will discuss the use of the various anti-spam tools and other techniques available within MailMarshal SMTP, and provide instructions for their implementation. OPTIMUM BASE CONFIGURATION To achieve the highest catch rate and optimum performance, anti-spam rules should be run in the order of the MailMarshal 6.7 default rules: SpamBotCensor and SpamProfiler SpamCensor and SpamProfiler SpamProfiler SpamBotCensor SpamCensor DNS blacklists When SpamProfiler, SpamBotCensor, SpamCensor, and DNS blacklists are used in this order, most environments will see a spam catch-rate of over 99.5%. This document will cover more than just the aforementioned rules, but at a bare minimum these should be enabled. Additional functions such as URLCensor and CountryCensor can improve performance further. Notes: For basic information about these essential rules, see the Anti-Spam Basics white paper. If you are reviewing your configuration after upgrading from a previous version, you should read the latest Default Rules document and consider updating rules to match the new defaults. In order to maximize the effectiveness of the anti-spam components, the primary MX record for a domain should point directly to the server on which MailMarshal SMTP resides, rather than directing through a forwarder, SMTP proxy, or relay of any sort. The direct connection is required for use of SpamBotCensor. Many powerful checks used by SpamCensor and the other rules depend upon a remote host s initial communication with MailMarshal SMTP. Ensuring a direct MX connection is essential to ensure the effectiveness of any DNS Blacklist checks that are performed within Receiver rules, because the blacklist checks query the IP address of the connecting host. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 2

3 SPAMPROFILER The SpamProfiler is MailMarshal s signature-based method of spam detection. In order to keep up-to-date with the latest outbreaks, signature updates are provided as frequently as every minute. The SpamProfiler feature is available in MailMarshal 6.4 onwards. Given that the SpamProfiler is a signature-based system with very frequent updates, it is imperative to make sure the updates are being received correctly. Unlike the SpamCensor updates which are performed centrally on the Array Manager, the SpamProfiler updates are performed locally by each node. The Service responsible for the updates is the MailMarshal Receiver service, and as such, signature update logging and troubleshooting information can be found in the Receiver service logs. If the signatures are being updated normally you should see entries like this in the Receiver logs: SpamProfiler: [MICROUPDATE] Successful signatures incremental download from network The downloaded signature files can be found on each node under MailMarshal s install folder by default, the full path is: C:\Program Files\Marshal\MailMarshal\SpamProfiler\micro_updates\ The two main signature files have a file extension of.aes and the expected size is 10s of MBytes Using the SpamProfiler The SpamProfiler is enabled and configured in the MailMarshal Configurator. From the Tools menu, click MailMarshal Properties > Receiver Properties > SpamProfiler. Basic configuration of SpamProfiler is through this interface. You can also use the SpamProfiler result in rules (recommended and provided in the default policy in version 6.7). The following options are available when using the SpamProfiler: Deny at Receiver. The SpamProfiler can identify spam messages at SMTP connection time. The message must be received in order to be matched against the SpamProfiler signature database. Before the SMTP transaction is completed, MailMarshal will return a permanent error to the sending server to indicate rejection of the message. While this does not provide any clear bandwidth savings (the message is received), it does eliminate the need for any further handling of the message. Any legitimate mail server blocked in this way will be obliged to send a notification to the original message sender. Do not deny at Receiver. If the above option is not selected, SpamProfiler results will be saved with each message. The results will be available for use in Standard rules. Exclude messages. You can choose to exclude specific messages from SpamProfiler evaluation. You can apply a global whitelist as well as per-user safe lists (configured through the Spam Quarantine Management website), and you can exclude outbound messages. All of these options can help to reduce false positives. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 3

4 SPAMCENSOR SpamCensor is an advanced heuristic filter that utilizes a combination of techniques to identify spam. Much more than a simple key word filter, it utilizes the following: Detailed header analysis. This technique closely examines message headers for any indication that it may be spam. The SpamCensor looks for hundreds of typical spam indicators. These include irregularities such as missing To or From header fields, invalid dates, and spaces in unusual places; they also include typical traits or spam genes usually left by bulk mailers or spamware the tools used to create and send spam. Advanced analysis of message content. The SpamCensor filter performs advanced searches of message content. It searches for thousands of spam patterns, including common phrases that promise get rich quick schemes and a better sex life, words with gaps between the letters, and sophisticated HTML patterns known to be associated with spam. It has rules that target different areas of each message, including plain text, raw html, and URL links. It can scan anything from the text between HTML tags, to the contents of the HTML tags themselves. Message composition. SpamCensor checks the message size and composition. Spam is not typically large, and often has only an HTML part. This information is used alongside numerous other indicators. As the SpamCensor runs, the results from each of the thousands of tests contribute to an overall spam picture. Each item contributes to a numeric score. Once the score exceeds a threshold, MailMarshal SMTP will treat the message as spam and take a predefined action. This weighted score approach results in high spam detection rates with few false positives. Using the SpamCensor The following files are referenced in the discussion of SpamCensor: SpamCensor.xml SpamChecker.dll SpamEvals.dll spamfilter.xml UserDefined.xml These files are found in the Config directory within the MailMarshal SMTP installation path. In a fresh installation of MailMarshal SMTP the installation path is C:\Program Files\Marshal\MailMarshal\Config\ Before using the SpamCensor functionality, you should ensure that MailMarshal SMTP is using the latest revisions of these files by performing a Spam Update (in the Configurator, see Tools > Server and Array Properties > Spam Updates). If the updater is unable to check for updates, please contact M86 Security Technical Support. Basic Configuration SpamCensor is designed to be simple to set up, and once enabled in a rule it will immediately begin catching spam. For the most basic configuration, see the default rules as described in MailMarshal Anti-Spam Basics. In MailMarshal SMTP 6.7, these rules use the condition Where message is detected as spam by If your MailMarshal installation was first installed with an earlier version, you should consider updating your rules to use this condition. Review the Default Rules document to understand how this condition is used. Although there is a range of more advanced adjustments that can be made, in most cases the basic configuration is all you need. Category Scripts Category Scripts are XML configuration files which contain different types of rules for checking . The SpamCensor is a special type of Category Script. This section discusses basic configuration of the SpamCensor as a Category Script. Invoking the Category Script allows you to use the filter by type function. Note: Before creating a SpamCensor rule, check that one does not already exist. To enable the SpamCensor as a Category Script: 1. Start the Rule Wizard by right-clicking an existing Policy Group, and selecting New Rule. 2. Choose Standard Rules and select Next until you arrive at the Rule Conditions window. Select the checkbox Where message is categorized as. 3. Create a rule that uses the SpamCensor.xml file. You will see a window as below. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 4

5 You will notice in the window that there are other XML Category Script files to choose from, some of which may not exist within your installation. These may include other standard scripts, and custom scripts. Warning: In most cases you should select only a single Category Script in each rule. Multiple Category Scripts should NOT be checked in this window. Selecting multiple scripts significantly reduces the catch rate of a rule. If multiple scripts are selected the rule is evaluated using an AND condition. ALL selected scripts must return positive for the condition to be true. For example in this instance, if both SpamCensor AND Spamhaus were checked, the rule would only evaluate true for messages that trigger BOTH on SpamCensor, AND on IP addresses blacklisted on Spamhaus. 4. You should finish with a rule that looks similar to this: Standard Rule: Block Spam - SpamCensor Category Script When a message arrives Where message is incoming Where message is categorized as 'Spam' Move the message to 'Spam' You can use all the usual rule elements to refine your rules. For instance, you can combine a whitelist and a size rule to improve accuracy. The whitelist would typically contain lists of newsletter sources, or other trusted or key sources of bulk . MailMarshal SMTP can even be configured to automatically generate a whitelist of friendly senders by harvesting recipient addresses on outbound s. Size conditions could be added as well, which would eliminate scanning of larger s that are unlikely to be spam. Note: Any refinements, particularly size conditions, should be reviewed regularly to ensure they are not reducing the effectiveness of SpamCensor. Default spam rules in MailMarshal 6.7 do not include size conditions (this is a change from earlier versions). A note on False Positives Whitelists are an important tool to reduce false positives. The SpamCensor is a heuristic filter that seeks to identify unsolicited bulk . Wanted bulk can be difficult to distinguish, since users may disagree about which messages are wanted. A comprehensive list of friendly addresses not only ensures the successful receipt of wanted , but also has the additional benefit of allowing the filters within MailMarshal SMTP to be stricter than is feasible within a default setup. Note: Over-use of whitelists, especially the use of wildcards, can contribute to false negatives. In particular, whitelisting your own domain allows significant amounts of spam to pass through. Some automated tactics will be detailed later in the Whitelisting Practices section. Administrators should also encourage and train their end-users to make use of the web-based Spam Quarantine Management system. This system allows each user to create personal white and black lists. Since this document is primarily concerned with fine-tuning anti-spam filters, setup and configuration of the Spam Quarantine Management system is not covered here. Additional information regarding the Spam Quarantine Management system can be found in the MailMarshal SMTP User Guide, or by contacting M86 Security Technical Support. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 5

6 Reviewing the SpamCensor Result You can examine the log file in the MailMarshal Console for the reason why a particular message is blocked. In order to find the message in question, you can either attempt to locate the message in Mail History, or perform a search based upon relevant criteria. Once the message is found, click the tab labeled Log, and you will see an excerpt like the one below: SpamCensor Logging Levels By default, MailMarshal SMTP does not retain a record of the SpamCensor score for messages that are not blocked. When testing the SpamCensor it is sometimes useful to know what rules triggered when a message did not reach the trigger level. The following Registry setting causes the SpamCensor to always log its output. Open regedit on the Array Manager server, and navigate to: HKEY_LOCAL_MACHINE\SOFTWARE\Marshal\MailMarshal\Default\Engine Add the following registry entry: Name: LogSpamAlways. Type: DWORD Value: 1 Set the value to 1 (true) to enable this extra logging. Setting the value to 0 (false) will disable the extra logging. This setting does not significantly affect server load, but it does add volume to the text logs. To apply this logging change, commit MailMarshal configuration, and then restart the Engine service. SPAMBOTCENSOR SpamBotCensor leverages the evaluation technology of SpamCensor using M86 Security s research into spam sources and particularly the major botnets that are responsible for the majority of spam. SpamBotCensor is updated through the same update process used for SpamCensor. SpamBotCensor can efficiently identify a large percentage of spam using a smaller number of evaluations for each message. SpamBotCensor does not allow any advanced configuration. To use SpamBotCensor, see the standard rule condition Where message is detected as spam by When using SpamBotCensor, ensure that SpamCensor updates are working and also ensure that MailMarshal receives incoming mail directly from the Internet. URLCENSOR URLCensor queries external URL blacklists which provide records of domains that appear to be frequently advertised within spam messages. These lists work in a similar fashion to DNS IP blacklists, but differ in that they list URLs instead of IP addresses. The original purpose of this functionality was to provide a method of blocking messages that contained very few triggers other than a link to a notoriously spam-advertised domain. Over time it has proven to be an excellent complement to SpamCensor. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 6

7 How does URLCensor work? URLCensor parses the body of an , and extracts all the domain portion of any URLs that are found. It then performs a DNS A record lookup for each domain using the DNS blacklist. There are currently two permutations of URLCensor available for use within MailMarshal SMTP: URLCensor URLCensorIP URLCensor Checks domains found within a message body against a third-party DNS blacklist. By default URLCensor uses the blacklist maintained by SURBL.org, or more specifically, multi.surbl.org. It can easily be configured to use other blacklists as well (covered further on in this document). multi.surbl.org is a combined zone utilizing domains provided by SpamCop, abuse-butler, SpamAssassin, and others. If for example URLCensor were to query marshal.com against SURBL.org, it would query the A record for marshal.com.multi.surbl.org. By default, if the DNS query returns any record at all, URLCensor will consider the domain to be blacklisted. If no record is returned from the blacklist s DNS server, the domain is not considered to be blacklisted. Once the lookup is performed, the result, whether positive or negative, will be cached by URLCensor for a certain (adjustable) time to preserve performance and avoid the need for repeated DNS lookups. URLCensorIP Performs in a similar way to URLCensor, but is designed to query against a blacklist that is formatted by the IP address of the A record for the domain, rather than by the domain name. URLCensorIP resolves the domain to an IP address using a traditional DNS query, and then submits the DNS blacklist query. By default, URLCensorIP uses the combined Zen combined blacklist maintained by Spamhaus. Again if any result is returned, the domain is considered to be blacklisted. If no record is returned, the domain is not considered to be blacklisted. URLCensorIP caches the results of these queries for a specific interval in case they need to be used later. Querying the IP address instead of the domain name is useful because spammers register large numbers of new domains, and thus domain blacklists are difficult to keep up-to-date. However, because the spam-related domains typically use a much smaller number of IP addresses, it is easier for the IP based blacklists to maintain a good hit rate. Both URLCensor and URLCensor IP can be configured to query other third party blacklists, so long as they are in one of the two supported formats. Both also have a configurable cache duration. For more information, see the White Paper MailMarshal SMTP Anti-Spam Advanced Configuration. Using the URLCensor The following files are referenced in the discussion of URLCensor: SpamSurbl.dll URLCensor.xml URLCensorIP.xml These files are found in the Config directory within the MailMarshal SMTP installation path. In a fresh installation of MailMarshal SMTP the default installation path is C:\Program Files\Marshal\MailMarshal\Config\ Before using the URLCensor functionality, you should ensure that MailMarshal SMTP is using the latest revisions of these files by performing a Spam Update. If the updater is unable to check for updates, please contact M86 Security Technical Support. The URLCensor is intended to be simple to implement. This section discusses basic configuration of the URLCensor. In most cases the basic configuration is all you need. However, for those who like experimenting, a range of advanced adjustments can be made (see the White Paper MailMarshal SMTP Anti-Spam Advanced Configuration ). To enable the URLCensor, create rules that use the URLCensor and URLCensorIP Category Scripts. In new installations of MailMarshal these rules are present by default. These rules make use of the same Categories Rule Condition that is used for SpamCensor and all other Category Scripts. To enable URLCensor: 1. Start the Rule Wizard by right-clicking an existing Policy Group, and selecting New Rule. 2. Select Next until you arrive at the Rule Conditions window. 3. Select the checkbox Where message is categorized as. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 7

8 4. Create a rule that uses the URLCensor.xml file. You will see a window as below: 5. Select the checkbox for URLCensor.xml 6. Complete the wizard by naming the rule, and deciding upon an appropriate action. Many actions can be taken based upon company policy and what the Administrator deems appropriate. For instance you can move the message to a folder, or simply flag the message for handling by the end-user s mail client. You should finish with a rule that looks similar to this: Standard Rule: Block Spam URLCensor (by Domain) When a message arrives Where message is incoming Where message is categorized as 'URLCensor Blacklisted' Move the message to 'Spam' As with SpamCensor, you can use all the usual rule elements to refine your rules. You can add a whitelist, a size rule, a TextCensor excluding certain domains, and so on. To enable URLCensorIP: 1. Start the Rule Wizard by right-clicking an existing Policy Group, and selecting New Rule. 2. Select Next until you arrive at the Rule Conditions window. 3. Select the checkbox Where message is categorized as. 4. Create a rule that uses the URLCensorIP.xml file. You should finish with a rule that looks like this: Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 8

9 Standard Rule: Block Spam URLCensor (by IP Address) When a message arrives Where message is incoming Where message is categorized as 'URLCensor IPBlacklist' Move the message to 'Spam' URLCensor and False Positives Whitelists always play an integral role in preventing false positives. However, because of the nature of the URLCensor checks, false positives are very rare. The team at Spamhaus and SURBL do an excellent job of maintaining these lists and keeping them up to date to ensure that legitimate domains do not get listed, and that spam-advertised domains are listed in as timely a manner as possible. Reviewing the URLCensor Result You can examine the log file in the MailMarshal Console to determine the reason why a particular message is blocked by URLCensor. You will see an excerpt like the one below. The log file illustrates how the URLCensor works. From this log, you can see that the domain (j4fimage.com) is blacklisted on multi.surbl.org. This particular domain exists in the DNS blacklist maintained by SURBL.org, indicating that, SURBL.org had received indications that this domain was a commonly spam-advertised domain. REPUTATION SERVICES (DNS BLACKLISTS) In addition to examining domain names (URLs) found within a message body, MailMarshal SMTP can examine the list of servers through which a message has traveled to see if any of them are known spam sources. The IP addresses found within Received lines of a message header indicate the servers through which a message has traveled. As the services use DNS as the method of querying their servers, they are also often referred to as DNS blacklists. Marshal IP Reputation Service With MailMarshal SMTP 6.7, M86 Security introduces the Marshal IP Reputation Service, a DNS blacklist based on information gathered by M86 Security and available exclusively to MailMarshal customers. For more information about this service, see the User Guide. Other Services There are quite a number of blacklists available on the Internet. The lists vary in quality, availability, and aggressiveness of listing policies. These lists are usually maintained by non-profit organizations, although some charge for certain services. One service that has a long history of accuracy and reliability is Spamhaus ( Note that use of Spamhaus services may require payment. Please carefully read the information at Each of the various blacklists has its own criteria for determining the contents of their respective lists. Before adding a new DNS blacklist, you should read the listing policy, if it is public, and speak to other users to determine the likelihood of false positives. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 9

10 MailMarshal SMTP can perform queries against these blacklists, to aid in evaluation of messages. MailMarshal SMTP provides rules to query the Marshal IP Reputation Service and Spamhaus by default. If your installation has been upgraded from an older version, you may need to configure these services and create the rules. This section will deal solely with the Marshal IP Reputation Service and Spamhaus. Other lists can be easily integrated. This process is discussed in the White Paper MailMarshal SMTP Anti-Spam Advanced Configuration. How can MailMarshal SMTP use DNS Blacklists? MailMarshal SMTP can perform DNS blacklist lookups of IP addresses in two ways: Receiver Rules Standard Rules (using Category Scripts) Each of these two methods has a specific purpose. For optimal performance and anti-spam recognition, you can complement a Receiver rule that performs a DNS Blacklist lookup with a Standard rule that does the same. The reasons for this recommendation are explained below. DNS Blacklists in Receiver Rules and Standard Rules There are important differences between the behaviors of Receiver rules and Standard rules in the use of DNS blacklist lookups. Both rule types have their merits and drawbacks, and ideally both should be used. Receiver-based DNS blacklist lookups Receiver rules that utilize DNS blacklist lookups query the IP address of the connecting host. This is one of a number of reasons MailMarshal SMTP should be the gateway of the network (the first server that accepts a message when it enters the network). Receiver-based DNS blacklist lookups are rendered useless if another gateway is placed ahead of MailMarshal SMTP. In that case the connecting IP address will always be the same (the IP of the other gateway). The IP address of the external server that connected to this gateway might be blacklisted, but the MailMarshal Receiver has no information beyond the server that connected directly to it. Even if a message originated from a known spam source, a Receiver rule will never trigger because the connecting server is NOT listed as a known spam source. Another issue can occur if the MTA of your ISP is designated to handle inbound prior to passing it on to MailMarshal SMTP. In the rare event that the MTA of your ISP is listed on a DNS Blacklist, all will be rejected by the Receiver. Because Receiver rules reject a message rather than simply quarantining it, if a legitimate message is inadvertently rejected at the Receiver, it will never be retried but is returned to the sender immediately. This threat is remote but should be considered prior to enabling this or any type of Receiver rule. On the other hand, this same behavior can provide an excellent benefit in terms of bandwidth and performance. A Receiver rule will reject a message subsequent to the remote MTA issuing the RCPT TO command in the initial SMTP handshake. In this scenario, the actual message body is never transmitted. The benefits of this are twofold: Rejecting a message prior to the sending of the message body can reduce the bandwidth consumed by unwanted, unsolicited messages. Preventing the message from entering the system also prevents it from consuming a MailMarshal Engine thread. Typical installations will have 2 Engine threads (with 4-5 in extreme circumstances on more robust hardware). Any message the Engine does not have to deal with improves performance, and frees the Engine to appropriately process legitimate messages entering the system. Standard Rules performing DNS Blacklist lookups Due to the limitations of Receiver rules, in most instances they should be supplemented with Standard rules. Standard rules use Category Scripts to perform DNS RBL lookups against lists. Standard Rules using RBL lookups cause MailMarshal SMTP to parse through the Received lines within a message header for IP addresses of servers. Each IP address found results in a query to the DNS RBL. This method of DNS RBL lookup implementation has the benefit that it checks for blacklisting of intermediate servers through which a message has traversed. If any of these servers are listed, the IP address will trigger the rule. However, since the entire message is received before a Standard rule is applied, the bandwidth to transmit the message has already been used and an Engine thread will be required to process the message. Important Note on DNS Blacklist Lookups URLCensor, URLCensorIP, Marshal IP Reputation Service, and Spamhaus all require frequent requests to be sent to DNS. The DNS server used for these lookups, as well as any other functions within MailMarshal SMTP that require DNS, is the DNS server specified within the Delivery settings in the Configurator. If an array of MailMarshal SMTP Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 10

11 servers is in use, by default all nodes in the array will use the Delivery settings defined in the Server and Array Properties. You can specify custom Delivery settings for a node in its individual Server Properties. MailMarshal SMTP will NOT use the DNS server specified in the TCP/IP settings of the server s network interfaces at any point in the message handling. It is absolutely imperative that a responsive, dependable, and forward-resolving DNS server is specified within MailMarshal SMTP s Delivery settings. One of the most common causes of poor Engine throughput and Receiver responsiveness is a setup that uses DNS lookups through a slow or unresponsive DNS server. This problem is especially noticeable when DNS blacklists are used. You can check the time used for each processing action by reviewing the text Engine log. If the DNS server seems to be a source of delay, you may wish to set up an internal, local DNS server using a DNS Zone Transfer for the DNS blacklists in question. The procedure for setting up this configuration lies outside the scope of this document, and will vary depending on the DNS server software being used. DNS Blacklist Server Downtime and Timeouts Occasionally DNS Blacklist servers become unavailable. In this scenario, MailMarshal SMTP waits for a period after a failed DNS Blacklist connection and tests connectivity before resuming full use of the server. Messages will be processed without checking against the DNS Blacklist until the server becomes available again. By default MailMarshal SMTP re-tries a server four times before marking it unavailable. Configuring Blacklists To configure Blacklists (version prior to 6.4): Before you enable any Receiver rules that use DNS blacklist lookups, in MailMarshal versions below 6.4 you must enable each blacklist within the Host Validation window on the MailMarshal Configurator. For details of this setup, see the User Guide and Help for your version of MailMarshal. To configure Blacklists (version 6.4 and above): 1. In the left pane of the Configurator, expand Reputation Services. 2. To add a service, click New Reputation Service. 3. Complete the Wizard. See Help for detailed information about the fields. For a generic service, enter the domain to query (see the documentation for the specific list). For instance, to add an entry for Spamcop enter bl.spamcop.net For the Marshal IP Reputation Service, enter the Customer Number and Activation Code related to your MailMarshal Product Key. If you do not have this information, you can retrieve it from the M86 Security website using the link provided on the wizard. RBL use within Receiver rules To create a DNS Blacklist Receiver rule (all versions): 1. Start the new rule wizard by right-clicking the desired policy group and selecting New Rule. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 11

12 2. On the first pane, change the type from Standard Rule to Receiver Rule and then click Next. 3. If a whitelist of safe senders is available, it should be used. To use a whitelist, on the User Matching pane, add the User Matching condition Except where addressed from. Then click the red users hyperlink and select the user group corresponding to the whitelist, as seen below: Once the group is highlighted, click the middle double - arrow (<<) to add the user group, then click OK. Using a whitelist excludes friendly senders on the list from having mail rejected by this rule. NOTE: Due to the aggressive nature of Receiver rules, it is good practice to exclude a list of known legitimate senders from Receiver rules in general. If a Receiver rule is triggered, MailMarshal SMTP will respond with a 500 series response code, which means that the message is rejected permanently. This code will cause the connecting server to generate a Non Delivery Report (NDR) and return it to the original sender. 4. In the rule wizard, click Next. 5. Select the option Where sender s IP address is listed in Reputation Service (earlier versions: DNS Blacklist. ) 6. On the blacklist selection window, all available DNS Blacklists are listed. Check the box to select the DNS Blacklist of your choosing and then click OK. 7. Click OK to continue to the Rule Actions pane. 8. Ensure that Refuse message and reply with is selected. You can customize the response code and brief message sent by clicking the blue Refuse message hyperlink. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 12

13 You should finish with a rule that looks like this: Receiver Rule: Deny Spamhaus Blacklisted Senders at Receiver When a message arrives Where message is incoming Except where addressed from Global Whitelist Where sender s IP address is listed in 'Spamhaus Zen' Refuse message and reply with 'Rule imposed as {Sender} is blacklisted on Spamhaus (see Using Blacklists within Standard rules Through the use of Category Scripts, MailMarshal SMTP can utilize DNS Blacklists within Standard rules. MailMarshal SMTP 6.7 includes rules to use Marshal IP Reputation Service and Spamhaus. If these blacklists are not currently in use, setting them up is as quick and simple as utilizing any other Category Script, such as SpamCensor. To enable Marshal IP Reputation Service and Spamhaus checks within Standard rules: 1. Start the Rule Wizard by right-clicking an existing Policy Group and selecting New Rule. 2. Select Next until you arrive at the Rule Conditions window. 3. Select the checkbox Where message is categorized as. 4. Create a rule that uses the appropriate XML file. You will see a window as below. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 13

14 You should finish with a rule that looks similar to this: Standard Rule: Block Spam Spamhaus Blacklisted When a message arrives Where message is incoming Where message is categorized as 'Spamhaus Blacklisted' Move the message to 'Spam' Reviewing the Standard Rule Results Using the MailMarshal Console, you can examine the message log file to determine why a particular message was blocked by either the standard blacklist rules. Note: Messages rejected by Receiver rules will not be shown in the MailMarshal Console. Analysis of receiver rules will require manual review of the MMReceiver logs. For messages blocked by Standard rules, you will see an excerpt in the Console like the one below: In this instance, note that the IP address being queried ( ) was not listed on sbl-xbl.spamhaus.org, but, it was listed on bl.spamcop.net (the DNS query to bl.spamcop.net using this IP address returned a record). The log shows the IP address in reversed order. This is simply due to the setup of most IP-based DNS Blacklists. COUNTRYCENSOR Included with MailMarshal SMTP is a powerful, unique utility called CountryCensor. CountryCensor allows mail administrators to identify the countries through which a message has traveled, and handle it accordingly. This capability can be very useful for an environment that receives little legitimate from countries other than its own, or for environments where from specific countries should be handled in a manner different from others. It is important to note that CountryCensor does NOT look at the top-level domain name found in any part of a message but rather examines the IP addresses in the message header to determine the countries through which the message has traveled. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 14

15 Prerequisites CountryCensor requires the following files be in place within the Config folder under the MailMarshal SMTP installation path: CountryCensor.xml CountryCensor.dll CountryCensorGroups.xml ip.db cc.db A default installation of MailMarshal SMTP will already contain these files. If they are not present within your installation, please contact M86 Security Technical Support. Basic Configuration CountryCensor currently requires some manual configuration. With the assistance provided in this document, the configuration should prove relatively straightforward. Prior to enabling CountryCensor within a rule, you must configure it. All configuration takes place within CountryCensor.xml. There are two options for adding countries to be checked by CountryCensor: Adding the two-letter country code for a specific country Adding a META group, which includes all countries that reside within that region Two-letter country codes and their corresponding countries for use within CountryCensor are listed at the bottom of CountryCensor.xml. The countries included in each region and their corresponding groups are listed in CountryCensorGroups.xml. These files include many comments, and most of the options available are described in the files. To prepare CountryCensor to be used within a rule: 1. Launch a text editor (such as Notepad). 2. Edit CountryCensor.xml 3. Within the file, locate the Group entitled BlacklistedCountryCodes and add the desired two-letter country codes as seen below: If you wish to include all of the countries within a region, add the region here as well, see below: By default, BlacklistedCountryCodes includes a META group named TopSpammers, which is simply a group including the top thirteen spam-producing companies. This group is merely provided as a demonstration of how to create and use META groups. You may choose not to use it. In any case, it Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 15

16 should not be used until it has been modified to include or exclude whichever countries are suitable for your environment. You can safely add or remove two-letter country codes from this group and include it in your CountryCensor check by adding it to BlacklistedCountryCodes as shown below: IP addresses of servers that should be excluded from CountryCensor checks should be added to the section entitled CCBlacklistExclusions. Each IP address should be on a line by itself. 4. Once satisfied with the configuration options, save the file and close the text editor. When you have finished editing the configuration file, you can use CountryCensor within a rule. The sample settings illustrated above will cause CountryCensor to trigger on the following countries: United States United Kingdom New Zealand Australia North America including the following: (AG,AN,BB,BM,BS,CA,CR,CU,DM,DO,GD,GP,GT,HN,HT,JM,KY,LC,MQ,MX,NI,PA,PR,PY,SV,TT,US,VG,VI) TopSpammers including the following by default: (US,CN,ES,KR,FR,PL,BR,DE,RU,IN,IL,IT,GB) Using CountryCensor within MailMarshal SMTP The steps described in this section enable CountryCensor to trigger on a message that has traversed servers in any of the countries defined in CountryCensor.xml. To enable CountryCensor: 1. Start the Rule Wizard by right-clicking an existing Policy Group and selecting New Rule. 2. Select Next until you arrive at the Rule Conditions window and select the checkbox Where message is categorized as. 3. Create a rule that uses CountryCensor.xml. You should finish with a rule as seen below: Standard Rule: Block Spam CountryCensor Banned Countries When a message arrives Where message is incoming Where message is categorized as 'CountryCensor' Move the message to 'Banned Countries Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 16

17 Note: You should use this rule in conjunction with a comprehensive whitelist. While CountryCensor is a very powerful utility when configured properly, it makes no distinction between legitimate and unsolicited mail from a blocked country. An extensive variety of options and tricks can be used with the CountryCensor technology. You could decide to list (and block) a few countries that are known to be major spam producers. You could use it to define a list of allowed countries, then quarantining from all but the known friendly countries. HELO RULES MailMarshal SMTP can reject a message based on the validity of the connecting SMTP server. Spammers will frequently attempt to send your own IP address as their HELO name in an attempt to fool some older filtering systems. Per RFC specifications, a HELO name should be a server s fully qualified domain name as published in DNS. It should also match the connecting system s PTR record. MailMarshal SMTP now has the ability to reject a message solely based upon the HELO name used in the initial SMTP handshake. Creating this new type of rule simply requires creating a new Receiver rule. A typical HELO rule would look as follows: All of the typical Receiver rule options still apply. The options available for checking the HELO name are seen below: Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 17

18 Note: Use this condition with caution. As with other Receiver rules, it causes to be rejected permanently with no further notice. TEXTCENSOR SCRIPTS The easiest configurable addition to the default rules enabled within MailMarshal SMTP is the creation and use of TextCensor scripts within the existing rules. The MailMarshal Configurator provides a simple graphical interface for creating and modifying TextCensor scripts. By default MailMarshal SMTP includes a TextCensor script entitled Administrator maintained keyword list. If a rule is enabled to utilize this TextCensor script, an administrator simply needs to update the referenced TextCensor script. The changes to the configuration must then be committed before they will take effect. This allows the administrator to make immediate updates as they see spam messages missed by the current set of checks. For details of TextCensor options, see the User Guide and Help. Using TextCensor Scripts within Rules Once a TextCensor script has been created, it will then need to be referenced within a rule in order for its checks to be measured against messages. For example, if the Block Specific Spam rule is currently not created, the following steps can be taken to utilize the script: 1. Start the Rule Wizard by right-clicking an existing Policy Group and selecting New Rule. 2. Create a new Standard rule that reads as follows: When a message arrives Where message is incoming Where message triggers text censor script(s) Spam - Administrator Maintained Keyword list Move message to Spam Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 18

19 With this set up, when new spam variants come through that are not yet picked up by MailMarshal SMTP s definitions, administrators can simply add new entries to the referenced TextCensor script. This in turn will block the new spam variants before they become an issue. ATTACK PREVENTION MailMarshal SMTP provides settings that allow you to protect your system against Denial of Service attacks (DoS) and Directory Harvest Attacks (DHA). DHA attacks in particular are used by spammers to determine valid addresses at your domain. Directory Harvest Attack (DHA) Protection When enabled, DHA prevention guards your system against Directory Harvest Attacks (DHA). MailMarshal SMTP s DHA protection can detect a DHA, drop the connection from the connecting server and blacklist the server for a specified length of time. MailMarshal SMTP recognizes an attack when a remote server sends many messages to invalid users. Before enabling this feature, you must provide MailMarshal SMTP with a list of valid users. The easiest way to populate such a list is to import users from your mail server or Active Directory, using an LDAP or AD connector. See the section Whitelisting Practices, below, for more details. Setting up DHA Protection Setup for this feature is accessed through the Configurator under Tools > Server and Array Properties > Attack Prevention. Setup options and requirements differ slightly depending on the release of MailMarshal SMTP that is installed. Important Note: Before using DHA Prevention, you must provide MailMarshal SMTP with a list of all valid addresses within your organization. MailMarshal SMTP releases and earlier use a group entitled All Employees for this list. The All Employees group should NOT be renamed, nor should it be deleted. To use other groups, insert them into this group. MailMarshal SMTP release and above allow you to select one or more groups that contain the list of valid users. For details of the setup requirements for this function, please review the User Guide and Help for your installed version of MailMarshal SMTP. SPAM MANAGEMENT TIPS There are many different ways to handle messages once MailMarshal SMTP has identified them as spam. Header Rewriting MailMarshal SMTP has built-in header matching and rewriting ability. This feature can be used to tag the header to flag the message as spam. Then, instead of quarantining the message, it can be passed through to the end-user client where automatic rules can determine what to do with it. The message may, for example, be automatically moved to a Possible Spam folder for the end user to periodically review at their convenience. The following header rewriting configuration tags the subject line with [SPAM]. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 19

20 This rule is included within the default MailMarshal SMTP rules. If upgrading from an earlier version of MailMarshal SMTP, this can be used in a rule as follows: Standard Rule: Modify Subject Line of Spam When a message arrives Where message is incoming Where message is categorized as 'Spam' Rewrite message headers 'Rename Spam Subject' And pass message to the next rule for processing. You are not limited to rewriting the subject line. MailMarshal SMTP can also be used to add custom headers. For example, you may want to add a custom header field called X-MailMarshal and add Spam in the field. This has the advantage of keeping the subject line intact and the end-user s client (depending on the type) can usually be configured to detect its presence. The rule is as follows: Standard Rule: SpamCensor Flag Suspected Spam When a message arrives Except where addressed from 'Friendly Listservers' Where message is categorized as 'Spam' Rewrite message headers using 'Add X-Marshal Header' And pass message to the next rule for processing. To configure the custom header go to the Rule Wizard. In the Rewrite Message Header action, add a custom field as illustrated below. There are standards relating to header fields so ensure your fields start with X- and use only alphanumeric characters, see below. The second step is to add an entry to the field, in this case Spam. This is illustrated below. The header field will look like this: X-Marshal:Spam Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 20

21 You should finish with a rule that looks like the following: These approaches place the responsibility for handling a detected spam message on the client. This has the added benefit of allowing users to manage their own junk mail. It also ensures the mail administrator isn t responsible for the incurrence of any false positives. This next option alleviates the need for an end-user to undertake any mail client configuration. Tip: Sometimes users want to know what SpamCensor rules a message triggered. The logging result of the SpamCensor can be appended to an message with a message stamp. It can also be added to a message notification template, using a MailMarshal SMTP variable: {SpamCensorResult} In either the message stamp or notification template, type a { character to view a list of available variables, and select SpamCensorResult. Quarantining Detected Spam Rather than relying on client configuration, MailMarshal SMTP has the ability to quarantine a message at the server side. This is before it reaches the end-user s inbox. It is also the default behavior for most of the existing Anti-Spam rules within MailMarshal SMTP. In order to move a message to a folder rather than flagging it, you simply need to navigate to the Rule Actions pane of the new rule wizard. Then select Move message to folder. It is typically easier to move all spam messages to the same folder and the reason for this will be discussed later on in the document. However a quarantine rule would look similar to the following: When a message is quarantined to a folder, the user isn t required to do anything unless they deem a message to be legitimate. If the quarantine folder is set up to allow end-user spam management, the user can navigate to the Spam Quarantine Management website. They would also add the sender to their personal whitelist. If desired, the quarantine folder can be configured to send out a daily digest , informing the user what s have been quarantined during that day. Message Digests are generated on a per-folder basis so multiple folders mean multiple digests are sent to recipients. Additional configuration of the Spam Quarantine Management system is outside the scope of this document. For more information regarding the end-user Spam Quarantine Management system, please contact M86 Security Technical Support, or feel free to browse the knowledge base. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 21

22 WHITELISTING PRACTICES The use of a comprehensive whitelist of friendly senders is integral to ensuring reliable message delivery within any anti-spam system. While the filters contained within MailMarshal SMTP are highly accurate, the occasional false positive can require some fine tuning from the administrator. Whitelists are a must for any implementation of an antispam system. An extensive whitelist will help to prevent wanted from incorrectly being flagged as spam. This also has the added benefit of allowing spam filters to be more restrictive. The better the whitelist, the stricter you can be with from infrequent or unknown senders. This allows you to refine and restrict the anti-spam filters to extremely granular levels. Note: Be wary of including entire domains in a whitelist. You should not include your own domain or common domains such as *@hotmail.com. These domains are often spoofed, and whitelisting them can reduce antispam performance. Valid Recipient Whitelist Messages addressed to non-existent users in your domain are of no value, and cause valuable CPU, memory, and network resources to be consumed processing and delivering them. Each one of these incoming messages requires a connection to MailMarshal SMTP. Subsequently they use system resources when an Engine thread is occupied to scan the messages. In addition, once the messages have left the MailMarshal SMTP system they will be processed by the internal mail exchanger. The mail exchanger is usually configured to reject the message with a 500 series response code. MailMarshal SMTP, per RFC standards, is obliged to notify the original sender of the message that its delivery failed. This is done by generating an NDR (Non-Delivery Receipt). Most of the time, the invalid messages are spam messages, with a spoofed return-path. MailMarshal SMTP can be bogged down with attempting to send a number of Non-Delivery Receipts, especially to domains and senders that don t exist or innocent third parties that never actually sent the message to begin with. Sending illegitimate NDRs may well cause your server to be blacklisted. Having MailMarshal reject messages to non-existent addresses eliminates the need to generate and send NDRs. For this reason, MailMarshal SMTP should be given a list of every valid recipient for whom it should accept mail. The problem is, by default MailMarshal SMTP has no information about which recipients are valid within your domain. To generate a list of valid recipients, in most cases you can create one or more LDAP connectors that will import addresses from your environment (Microsoft Active Directory, or another LDAP directory). Note: You may also need to enter some addresses manually, if they are not readily accessible through LDAP. LDAP connectors can be configured to automatically update at specific intervals and so alleviate the need for an administrator to maintain the list. Once this list is imported, messages addressed to invalid recipients can be eradicated completely. In turn this decreases the load on MailMarshal SMTP and the backend mail exchanger. Ideally, a Receiver rule should be created to reject messages to invalid recipients. If a Receiver rule is used, the unwanted message will be rejected immediately after the RCPT TO command. This occurs during the initial SMTP handshake. Rejecting the message at this step in the process prevents the message body from ever being sent. This frees up bandwidth, engine threads and overall resource consumption - both on the server where MailMarshal SMTP resides and also on the internal server. The following procedure covers basic setup of a LDAP connector and user group, and a Receiver rule to reject messages based on the contents of this group. For advanced techniques to scrape every available address from Active Directory or other LDAP directories, see the white paper MailMarshal SMTP Advanced Anti-Spam Configuration. Step One: Setting up the LDAP Connector 1. On the MailMarshal Configurator, expand Policy Elements. 2. Right-Click Connectors and select New Connector. 3. On the Connector Type window of the wizard, select the appropriate type of connector, based on the server type. Click Next. 4. If you are connecting to Microsoft Active Directory, choose to connect anonymously or with credentials. Many installations require credentials to connect. Enter the appropriate information, and then click Next. 5. If you are connecting to another type of LDAP server, enter the server name and credentials as required, then enter additional information as prompted to specify the information that should be retrieved. To learn more about the settings, see Help for each window. Note: The wizard provides a selection of preconfigured connectors. If your LDAP server type is not in the Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 22

23 list, select Generic LDAP Server. You will be prompted for additional information. If necessary, ask the LDAP server administrator for the appropriate settings. 6. Set a reload schedule. MailMarshal will request updated information as scheduled. Click Next. 7. Enter a name and optionally a description for the connector. 8. Click Next, then Finish to complete the wizard and create the connector. Step Two: Importing Groups 1. On the MailMarshal Configurator, expand Policy Elements 2. Right-click User Groups and select New User Group. 3. On the User Group window of the wizard, choose Import one or more user groups from the connector. If you have created more than one connector, select the connector you want to work with (such as Active Directory). 4. On the Import User Groups window, click Browse to view a list of available groups. If necessary, ask the LDAP server administrator for the appropriate groups that contain user addresses. 5. Tip: In Active Directory, a Domain Users or similar group probably contains many user addresses. 6. Click Next, then Finish to complete the wizard. 7. Note the name of the group as created in the User Groups tree of the Configurator. 8. Right-click User Groups and select Reload User Groups to retrieve members from the group immediately. Select the group name to view a list of the addresses it contains. Step Three: Rejecting Mail to Invalid Recipients Once the valid recipient whitelist has been created, all messages addressed to invalid users can be filtered accordingly. The ideal method for accomplishing this is to create a Receiver rule that rejects messages outright when they have been addressed to an invalid recipient. Note: It is important to mention again that any addresses which do not exist in the list imported through LDAP or Active Directory must be manually added to the Valid Recipients group (either directly, or by being added to another group contained in it). This step is essential because a Receiver rule rejects the message rather than quarantining it. Any rejected messages will be forever lost. If you have doubts as to how comprehensive your valid user whitelist is, you may wish to set this rule up as a Standard rule first. Then have messages to recipients that aren t listed simply quarantined in a folder for you to review. This check can be set up as follows: 1. Expand Policy Elements and right-click User Groups. 2. Select New User Group and create a new MailMarshal SMTP user group named Valid Recipients. This group will be used to collect all LDAP and local groups that contain valid recipients. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 23

24 3. Right-click the newly created Valid Recipients group, and select Insert Groups. 4. Insert all groups created using LDAP and AD connectors. 5. Insert the MailMarshal SMTP user group that contains addresses that are not present in any LDAP or Active Directory groups. 6. Start the new rule wizard, and create a new Receiver rule that reads as follows: Receiver Rule: Deny Messages to Invalid When a message arrives Where message is incoming Except where addressed to 'Valid Recipients' Refuse message and reply with 550 Rule imposed mailbox access for {Recipient} refused: user invalid Note: There can also be a negative effect of rejecting to invalid recipients. Spammers can use the response from a Receiver rule to create a refined list of valid addresses within your organization. Addresses that give a 250 in response to the RCPT TO command are assumed to be valid. Addresses that return a 550 in this case, can be removed from the spammer s address list. This tactic is basically a Directory Harvest Attack (DHA). If DHA Protection is enabled within MailMarshal SMTP, rejecting messages to invalid recipients isn t especially needed. Automated Whitelisting To gather a useful whitelist, users can be trained to use the Quarantine Management System to whitelist their own legitimate senders, or gather addresses from their address books. As of MailMarshal SMTP version 6.x, the ability now exists to automate the generation of a whitelist in an intelligent manner. If an end-user sends a message to an address, there is a high probability that the address is not going to end up being a spam source. MailMarshal SMTP has the ability to harvest recipient addresses on outbound , for automatic inclusion within the Global Whitelist. Ideally, the top rule in an Anti-Spam rule-set would be a rule that skips over spam checks when the remote sender is in a group of known legitimate senders. By default this is named Global Whitelist. If you do not already have this group created, for organizational purposes, it is recommended to do so. The rest of this section will assume that there is an existing Global Whitelist User Group. With this set up, a typical top rule within an Anti-Spam rule-set would read: Standard Rule: Allow Senders in Global Whitelist When a message arrives Where message is incoming Where addressed from Global Whitelist Pass the message on to the next policy group Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 24

25 This is the simple part of the process. The difficulty is in obtaining an extensive, worthwhile whitelist. There are two crucial steps in getting this process working: 1. Exclude Postmaster bounces and Out of Office Replies from adding to the harvested list. 2. Set up a rule within MailMarshal SMTP to harvest addresses and add them to the Global Whitelist. Step One: Excluding Postmaster Bounces In order to exclude NDR messages from adding recipients to the whitelist, you initially need to set up a MailMarshal User Group that contains the common postmaster aliases. You should also include any custom postmaster/root aliases that might exist within your organization. For our purposes, we will create a new User Group entitled Postmaster Addresses. A typical list will look as follows: Though use of root as an alias within an organization is rare (and not recommended), you may also wish to add root@*.* Step Two: Excluding Out of Office Replies Out of office replies typically have few obvious characteristics that distinguish them from regular messages and as such will never be 100% detectable. However with the use of a TextCensor script, we can make a large percentage of these skip our harvesting rule. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 25

26 Create a TextCensor script similar to the following: Although there may be messages that inadvertently trigger this TextCensor script, the only potential problem once we are finished will be the failure to add the recipient to the whitelist automatically. Note: This script is intended to trigger on nearly every message and NOT trigger if it contains text common in out-of-office auto replies. You may wish to modify it to suit your own needs. Step Three: Harvesting Recipients 1. First, create a MailMarshal User Group (blank for now) entitled Harvested Whitelist 2. Next, right click your Global Whitelist user group and select Insert Group. Insert the recently created Harvested Whitelist group into the Global Whitelist group 3. Create a rule within your top outbound ruleset that reads as follows: As time goes on, this list will grow quite large. It does allow administrators to enforce a strict policy without concerns about legitimate messages being inadvertently trapped by their filters. You may wish to purge this list a few times per year and eventually lead the end-users towards taking advantage of the Spam Quarantine Management system. SUBMITTING MISSED SPAM OR FALSE POSITIVES TO M86 SECURITY In the event you receive an unacceptable number of spam messages, you may wish to notify M86 Security of the missed messages. Similarly, if a valid message is blocked, you may wish to notify M86 Security. Although it is easy to forward a message from the user s mail client, samples submitted in this manner offer limited useful information. The problem is that SpamCensor is optimized to check details that are lost when a message is simply forwarded. For instance, forwarding these messages loses much valuable header information, used by a Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 26

27 considerable number of the checks within SpamCensor. Even if the header information is pasted within the , forwarding still loses evidence of format items such as bare carriage returns and arbitrary spacing. In MailMarshal 6.7, you can submit a message as a false positive or false negative with a single click from the Console. The submission buttons are present on the message window (highlighted below) and also in the various mail listings where items can be multi-selected. In earlier versions of MailMarshal SMTP, you can forward a message from the Console to one of the following addresses: spam@m86security.com (missed spam) or notspam@m86security.com (false positives). Forwarding from the Console retains information and formatting that may be changed when you forward from a mail client. M86 Security attempts to gather an accurate picture of our customers message flow on an ongoing basis, particularly in regard to spam or viruses that slips through customers filters. Samples submitted to the Security Labs are exceptionally useful in our efforts to update our technology and to meet the needs of our customers. M86 Security respects its customers privacy and will not disclose any information contained within a false positive submitted for analysis. COMMON PITFALLS MailMarshal SMTP s effectiveness at blocking spam can be reduced by a number of internal and external factors. Three of the most common are: Selecting multiple category scripts within one rule this causes the Where message is categorized as category condition to evaluate each category as an AND. If for example, you selected both Spamhaus and SpamCensor, the message would have to have an IP address blacklisted on Spamhaus, AND trigger the default SpamCensor. This rule would catch significantly fewer messages than either one of the conditions alone. You might choose this combination intentionally to identify messages with high confidence, but in this case you would want to use other rules as well. For examples of multiple condistions used in this way, see the MailMarshal 6.7 Default Rules. Putting gateways in front of MailMarshal SMTP As mentioned previously, many powerful rules within MailMarshal SMTP examine the behavior of the connecting server. If this is your ISP s SMTP server or another relay host in front of MailMarshal SMTP, these checks will not trigger. Firewalls with SMTP proxies or SMTP proxying applications firewalls and proxies that attempt to fix SMTP traffic by dropping packets or removing headers are notoriously detrimental not only to the spam catch rate but to SMTP in general. Removal or modification of headers by upstream systems can lower the effectiveness of many spam checks, especially SpamCensor. Using entire domains in whitellists (such as *@hotmail.com) it is common for these domains, and also your local domain, to be spoofed. Whitelists should be as specific as possible. Technical Whitepaper: MailMarshal SMTP Anti-Spam Configuration Page 27