Messaging Anti-Abuse Working Group (MAAWG) Message Sender Reputation Concepts and Common Practices

Transcription

1 Messaging Anti-Abuse Working Group (MAAWG) Message Sender Reputation Concepts and Common Practices Abstract Reputation is commonly defined as a measure of whether the populace at large has a generally favorable or unfavorable opinion of a person or entity. In Internet messaging, a sender s reputation reflects how responsible and trustworthy a person seems to be based on a variety of characteristics. Unlike the common definition, however, this form of reputation is based more on hard data than arbitrary interpretations and widespread opinion. Reputation assessors which could be a person, organization, or automated software agent - perform the necessary activities to collect that data and calculate sender reputation. This process is sometimes accomplished inhouse directly by the message service provider or performed by a contracted third-party. Message service providers can then refer to the assessment results to decide how to handle a sender s messages with the goal of keeping the message recipients safer and happier. This white paper describes the inherent concepts and common practices concerning the reputation systems used in Internet messaging but it is not a specification or a set of requirements for all reputation systems. The paper is intended for a general industry-related readership and assumes basic familiarity with Internet messaging systems. Contents ABSTRACT... 1 EXECUTIVE SUMMARY...2 INTRODUCTION...3 ASSESSMENT GOALS...3 RISK... 3 TRUST... 4 TRUST VS. RISK... 4 ASSESSMENT MODELS...5 THE BEHAVIORAL ASSESSMENT MODEL... 5 THE CERTIFICATION ASSESSMENT MODEL... 6 COMPARING THE TWO MODELS... 6 ASSESSMENT ALGORITHMS...7 COMMON INPUTS... 7 CALCULATION... 8 COMMON OUTPUTS AND RECOMMENDED ACTIONS... 8 CURRENT USE...8 FUTURE USE...9 CONCLUSION GLOSSARY REFERENCES Messaging Anti-Abuse Working Group MAAWG Messaging Anti-Abuse Working Group P.O. Box San Francisco, CA info@maawg.org

2 Executive Summary It takes many good deeds to build a good reputation, and only one bad one to lose it. - Benjamin Franklin Reputation is commonly defined as the estimation in which a person or thing is held. It is a measure of whether the populace at large has a generally favorable or unfavorable opinion of that person or entity. Brands take great care to establish and maintain their reputations in the minds of their customers. Factors such as quality, convenience and dependability all affect customer opinion. Reputation is usually defined within specific contexts. Therefore, a good reputation in the business world or in the minds of consumers does not necessarily carry over directly into Internet messaging. While a brand s good reputation within its general social environment might provide a small boost to its online standing, it is the brand s online behavior and the opinions of its message recipients that contribute to their Internet messaging reputation. Message service providers use Internet messaging reputation to provide a better user experience and reduce operating expenses. Messaging reputation predicts whether a sender s future messages will be abusive or not. Message service providers can make deliverability decisions based on the sender s reputation quickly, reducing message handling costs. Message filters can skip expensive filtering steps on messages from reputable senders, avoiding false positives and user frustration over s mistakenly sent to the junk folder, while efficiently rejecting messages from disreputable senders at the edge of the network. Message filtering based on reputation is more robust and flexible than traditional whitelisting and blacklisting. The wide range of reputation scores can be mapped to appropriately aggressive filtering actions. Senders have the option of engaging a reputation assessor a person, organization, or software agent that calculates a messaging reputation result in order to establish trust proactively. Otherwise the sender s reputation will develop organically as message providers and ISPs assessors passively observe their messaging behavior. Reputation is already a powerful tool for fighting messaging abuse. As more organizations adopt domain authentication mechanisms, such as DomainKeys Identified Mail (DKIM), Sender Identification Framework (Sender ID or SIDF) and Sender Policy Framework (SPF), reputation will become even more powerful as a way to establish greater trust. [For more information on authentication, see the MAAWG white paper Trust in Begins with Authentication ( Message system operators of all types should take steps to monitor and protect their reputation to ensure appropriate delivery of their messages. This may be achieved by monitoring and filtering outbound messages, ensuring the security of messaging infrastructure, receiving and acting on recipient complaints via feedback loops (FBL) from mailbox providers who offer them, and making use of third party reputation assessment and monitoring services. MAAWG Message Sender Reputation Concepts and Common Practices 2

3 Introduction Using online reputation is a powerful approach to combating Internet messaging abuse. Reputation predicts whether a sender s future messages will be abusive or not. Message service providers use the sender s reputation, the result of a quality assessment of the sender, to decide how to handle a message, instead of the qualities of the individual message itself. The two most common examples of data-driven reputation in the real world are credit scores and driving records. Credit agencies assess an individual s financial assets and history to arrive at a score. Financial institutions can then refer to the result of that assessment, the credit score, when deciding whether or not to lend money to the individual. Similarly, insurance companies do not give all drivers the same rate simply because they are licensed to drive. The insurer takes into account numerous factors about the driver, including the driver s record, to determine how high of a risk it would be to insure them and then sets the rate appropriately. In Internet messaging, the actors in a reputation system are analogous to these real word examples. There are reputation assessors who perform the same duties as a credit agency, assessing all the identities people, organizations, or any other accountable entity who attempt to send messages. Handling filters fill the role of the financial institutions, referencing the assessment results to make message delivery decisions. A person, an internal department, a third-party organization, or an automated software agent can all act as a reputation assessor. The largest message service providers tend to perform their own assessments internally. In those cases the assessor and handling filter are the same entity. Handling filters can also use the results of thirdparty assessors. Reputation is very flexible. Assessors can identify both trustworthy and malicious actors. Filters on both the sending and receiving side of message transactions can make decisions based on reputation. The assessment results can be very broad and general or very targeted and specific. Assessment Goals The goal of sender reputation assessment is to determine how trustworthy the sending identity is or to measure the risk that the identity s messages may cause distress or harm to recipients, or both. Understanding the differences among these goals is key to understanding the different ways in which reputation is assessed and applied. Risk The result of a risk assessment reveals the probability that future messages from a given identifier - a unique online string that refers to an identity, such as a domain name or IP address - will be abusive. Assessing risk is of great interest to the anti-abuse community. While malicious senders can produce an almost endless number of variations in message content, they are constrained to a much smaller number of sending identifiers. Determining the risk associated with a particular message stream allows filters to adjust their aggressiveness appropriately. As previously unknown vulnerabilities are exploited and malicious senders change their tactics without warning, the risk associated with message traffic can fluctuate greatly over a short period of time. Therefore, reputation systems that attempt to measure risk perform their assessments on either a continuous real-time basis or at very short intervals in order to provide the most accurate and up to date results. MAAWG Message Sender Reputation Concepts and Common Practices 3

4 Trust A trust assessment aims to determine how responsible the identity behind the identifier is and how capable the identity is at handling abuse issues that arise. Traditional whitelisting is an example of a trust assessment. As trust increases, any observed abuse may increasingly be considered as simply errors the sending identity can be relied upon to address rather than abusive behavior. While risk advises filters as to how aggressive they should be, trust advises filters as to what filtering mechanisms are appropriate to apply and which to ignore. For example, a handling filter might choose not to apply filtering mechanisms that control the rate an identifier can send messages, such as connection management or rate-limiting filters, to moderate or highly trusted identities. Yet it would continue to apply basic content filtering to identities at all trust levels. Obtaining a moderate level of trust could be as simple as providing valid contact information and requesting a feedback loop (FBL) of recipient complaints or other similar steps to differentiate a legitimate and responsible identity from a malicious and evasive one. Higher levels of trust might require a very rigorous security examination of the identity s physical infrastructure and any processes or policies involved in message submission and delivery or even some form of binding agreement. The nature of the identity can also play a role in trust assessments. For example, any organization whose messages are considered protected by law, such as political or religious organizations, might be considered highly trusted. Assessors also afford a high level of trust to government organizations, particularly law enforcement and public safety related groups. In contrast to risk, trust is a more stable value. It requires knowledge of the sending identity that is not likely to change at the same rapid rate as an attacker s tactics. Reputation systems that attempt to assess trust may do so in a discrete manner at longer intervals. Unfortunately, an entirely trustworthy identity may operate equipment which is susceptible to attack and control by untrustworthy actors. As a result, if the identity s behavior changes quickly and drastically, a risk assessment may temporarily override any trust assessment thereby reducing the potential harm caused by a security breach or similar issue. Trust vs. Risk Many reputation systems attempt to measure both trust and risk in an effort to provide the most accurate possible guidance to their clients. In those cases, reputation is the combination of organizational trust and message risk, as shown in Figure 1. Each reputation system will, of course, have its own specific policies, statistical thresholds and reputation levels. Depending on the capabilities of their clients, many will offer higher degrees of risk and trust granularity in order to fine tune and carefully target their recommendations. Figure 1: Organizational Trust vs. Message Risk At low trust levels, either little or nothing is known about the identity or it has an established history of irresponsible behavior. The default filtering mechanisms are applied and delivery is entirely dependent on the MAAWG Message Sender Reputation Concepts and Common Practices 4

5 assessed message risk. That is not to say that identities at low levels of trust are unable to deliver messages. If the volume and risk of messages is relatively low then few difficulties should be encountered. In the event of high-risk messages, identities at low trust levels are likely to be fully blocked, perhaps even permanently. Filtering becomes more targeted at medium trust levels, as the more trustworthy identities are exempted from the harshest default filtering mechanisms. High-risk messages might still result in strong filtering actions, but those actions are likely to persist for a shorter duration than they would against an identity at a lower level of trust. Identities at medium trust levels with a history of low-risk messages rarely experience delivery issues, if at all. Highly trusted identities are not usually subject to filtering, as any filtering actions would result in false positives. However, even highly trusted identities can be compromised and abused on rare occasions. Such identities are expected to address the issue swiftly. In the event of very high-risk messages, such as a phishing attack, the reputation assessor will recommend fully blocking the compromised identity until the issue is resolved. The difference between high trust and the lower levels of trust is that the assessor will immediately recommend full delivery once the highly trusted identity has addressed the issue. Identities at lower levels of trust can expect a period of heavier filtering following a similar compromise. Of course, a highly trusted identity that has constant abuse issues is not likely to remain highly trusted. Assessment Models Two models exist for assessing message sender reputation: the behavioral model and the certification model. At first glance, the two models are very similar. In both models, the author - the actor who creates the message content - sends a message through a responsible identity to a recipient. A handling filter decides whether or not to deliver this message to the recipient based on guidance from a reputation assessor, which might consult multiple reputation databases for assistance. Assessment and filtering can occur on both the sending and receiving side of the message transaction, or a third party vendor can perform assessments. However, the behavioral model acquires data reactively while the certification model requires active participation from the responsible identity. The Behavioral Assessment Model In the behavioral model, reputation assessors calculate a result in reaction to the messages an identifier sends. The result of this assessment is a prediction of the quality of future messages based on the observed quality of past messages. The assessor works in the background, basing its decisions on data from handling filters, feedback from recipients and any external data sources it deems appropriate perhaps even the output of other reputation systems. Figure 2: The Behavioral Assessment Model MAAWG Message Sender Reputation Concepts and Common Practices 5

6 The Certification Assessment Model In the certification model, the assessor plays more of a central role. The responsible identity initiates assessment by contacting the assessor directly. In this manner, the assessor can perform the assessment prior to the identity sending any messages. Moreover, the certification model allows the responsible identity to act cooperatively with the assessor to address any abuse issues on an ongoing basis. Figure 3: The Certification Assessment Model Comparing the Two Models Cooperation and interaction between the responsible identity and the reputation assessor allows the certification model to perform a more effective trust assessment than the behavioral model does. The responsible identity can provide information to the assessor that is not available in the behavioral model, such as documented polices or contact information, and the identity can work with the assessor to maintain their good standing. However, relying on the responsible identity to initiate and cooperate with the assessor causes scale problems. The responsible identity must both be aware that an assessment is needed and have the necessary resources to participate. Many identities will either lack the resources or not meet the high level of quality required. It is therefore unlikely that a certification model assessor will be able to provide comprehensive guidance to a handling filter. The behavioral model, on the other hand, makes no additional requirements of the responsible identity and places the entire burden of assessment on the assessor. While a responsible identity must cooperate with the assessor in the certification model, he cannot avoid assessment in the behavioral model. This makes the behavioral model more adept at dealing with malicious identities and assessing risk. The weakness of the behavioral model lies in its reactive nature. The behavioral model requires some amount of history in order to perform an assessment and therefore can lead to false positives and false negatives until the assessor collects sufficient data. As a result of the pros and cons of both models, many reputation systems are a hybrid of the two approaches. This allows the two models to act in a complementary fashion so that each can support and balance the other. Handling filters can then benefit from the results of both trust and risk assessments and make a well-informed delivery decision. MAAWG Message Sender Reputation Concepts and Common Practices 6

7 Assessment Algorithms The basic process at the heart of the reputation system is always the same, as shown in Figure 4, regardless of the algorithm used to calculate reputation. First, the assessor collects the relevant inputs and feeds them to the algorithm. The algorithm then performs its calculation and recommends an action to message handling filters. This may be done in an automated fashion, manually, or by some combination of the two. Figure 4: The Reputation Assessment Process Common Inputs Reputation systems with differing assessment goals will have different inputs contributing to their respective reputation algorithms. Risk assessments tend to rely mainly on objective message statistics, filtering results, and recipient feedback. Trust assessments may incorporate other, somewhat more subjective, elements such as the type of identity being assessed and the identity s documented policies and practices. Most of the industry expresses message statistics as a percentage or rate of relevant events compared to total message volume over a period of time. Commonly used relevant events include: Messages reported as false negatives by recipients (complaint rate) Filtered messages reported as false positives by recipients Messages attempted to nonexistent recipients (invalid recipient or bounce rate) Messages containing viruses or malware Messages judged as spam by handling filters Messages sent to spam trap addresses The most universal input is recipient complaints, often measured as a percentage of complaints to message volume over a period of time. This complaint rate metric is widely used by mailbox providers and service providers as a basic reputation assessment. Beyond the message statistics, inputs tend to be more binary in nature. Assessors might perform various security related tests of the identity s messaging infrastructure, such as open relay or proxy tests, and include the test results as binary pass/fail inputs. Additionally, assessors might require compliance with certain industry best practices related to abuse handling or mailing list hygiene. The output of other reputation systems, such as a third party whitelist or blacklist, might also be included as input to an algorithm. MAAWG Message Sender Reputation Concepts and Common Practices 7

8 Calculation The complexity of assessment algorithms varies greatly across the industry. The simplest are a single metric compared against various thresholds or a checklist of policy-related questions. The more complex algorithms involve sophisticated machine-learning equations with a dozen or more inputs. The exact details of the more complex algorithms are often considered very valuable intellectual property and are kept secret. Regardless of how tamper-resistant they might be, there is always a chance that a malicious actor could learn to manipulate the algorithm to gain a more favorable result if the algorithm details were known. In general, the algorithm compares the individual inputs against various thresholds to determine where the identity falls in the trust or risk spectrum. The assessor sets the thresholds based on its own research and will vary those thresholds over time as further research dictates. When multiple inputs are used, the algorithm combines the individual results to reach a single final result. An algorithm might also give greater importance to some individual inputs in the final result due to a higher degree of relevance, accuracy or reliability. Common Outputs and Recommended Actions The most basic output of a reputation calculation is inclusion in a whitelist, resulting in successful delivery of all messages, or a blacklist, resulting in all messages being blocked. These are very coarse-grained, binary results. The main issue here is that all identities do not fall neatly into the white or black categories. Providing only this very coarse level of output leaves handling filters with a significant grey area to evaluate without guidance. More finely-grained assessment results allow assessors to recommend more specific actions of appropriate aggressiveness to handling filters. In this manner, assessors can subdivide the grey area into various shades of grey and handling filters can achieve a greater filtering accuracy. These actions include temporary blocks of varying duration, varying degrees of content filtering, rate-limiting of varying restrictiveness, etc. In addition to the wide variety of filtering actions that an algorithm can recommend, it may also confer additional benefits beyond successful delivery of identities deemed to have sufficiently high trust and low risk. These benefits often involve the activation of user interface features that mailbox providers would otherwise disable by default to protect the privacy and safety of recipients. Most commonly this includes the rendering of rich media elements (images, video, etc.) and hyperlinks to external resources that are embedded in the message. At the highest levels of trust and lowest levels of risk, the handling filter might even mark the messages in a distinctive manner indicating to the recipient that the messages are safe and trustworthy. Similarly, at the lowest levels of trust and highest levels of risk, messages may be marked in a manner indicating that the messages are dangerous. Current Use Most mailbox providers make use of reputation to some extent in their messaging infrastructure, although the degree of use and level of sophistication varies greatly across the industry. IP addresses are the most commonly used identifier, as it is trivial for a malicious identity to forge any of the other identifiers in a message. However, assessments based on IP addresses can sometimes cause false positives and false negatives as ownership is transferred to a new identity or multiple identities all use the same IP address for message transmission. IP address-based assessments can also be redundant, as a single identity might use multiple IP addresses for message transmission. MAAWG Message Sender Reputation Concepts and Common Practices 8

9 The largest mailbox providers tend to have their own internally developed and operated reputation systems. In those cases the assessor and handling filter are the same entity. The algorithm can be customized so that its output is specific to the provider s filtering strategy and is tightly coupled to its handling filters. In general, the next smaller tier of mailbox providers - particularly North American cable operators utilize third-party filtering and reputation systems, combining and customizing various assessments to implement a strategy appropriate for their particular customers and business needs. Other providers approaches vary more widely, from extremely sophisticated customized machine learning algorithms to black box filtering appliances to filters and block lists tuned by hand daily. Reputation is also used on the sending side of message transmission. Mailbox providers and service providers assess the reputations of their users and clients. The results of those assessments are used to filter outbound mail in order to protect the provider s own reputation as calculated by other providers and assessors. Future Use Authenticated identifiers will come into more prominent use in the future. The use of domain authentication technologies, such as DomainKeys Identified Mail (DKIM), Sender Identification Framework (Sender ID or SIDF), and Sender Policy Framework (SPF) will provide domain-based identifiers that will not suffer from the same issues as IP addresses. The identifiers will be more stable over time and their granularity will be more aligned with their responsible identities. This will lead to both more accurate assessment results and greater opportunities to establish trust. Responsible identities must first deploy these new domain authentication technologies in their messaging infrastructure in order to gain any benefits, but this can be a challenging process. Any such major addition to infrastructure requires the resources to build and then maintain them. Additionally, there is the question of which authentication technologies and with what configurations and policies a given organization should deploy to gain the best reputation possible. Domain authentication will also bring new challenges for reputation assessors and handling filters beyond the required infrastructure changes. A single message may potentially contain multiple authenticated identifiers, each of which may have their own distinct reputations. This raises the question of what actions an assessor should recommend in the face of conflicting reputations. The industry will develop best practices around how to handle these cases over time based on real-world experience, and thus further discussion is left for a future paper on the topic. Trusted third parties are expected to provide identity class or membership data in the future. An identity s class denotes the organization type of the identity, such as an Internet service provider (ISP), university, or financial institution. Membership reveals an identity s affiliation with a group, such as U.S. Federal Deposit Insurance Corporation (FDIC) member banks. While this type of data may not be useful in a binary whitelist or blacklist, it should permit assessors and handling filters to treat different types of identities more appropriately rather than subjecting every identifier to the same assessment criteria and filtering mechanisms. Assessors can also use this raw membership data to confer appropriate amounts of trust along the chain. For example, a government agency might publish a list of all the valid identifiers used by members of Congress or a political party might publish a list of all the valid identifiers to be used by all of their candidates in upcoming elections. The assessor can then include that information in their assessments of the published identifiers without collecting the data directly from the member identities. MAAWG Message Sender Reputation Concepts and Common Practices 9

10 Conclusion Reputation is already a powerful tool for fighting messaging abuse. As more identities adopt domain authentication mechanisms, reputation will become even more powerful as a way to establish greater trust. Message system operators of all types should take steps to monitor and protect their reputation to ensure appropriate delivery of their messages. This may be achieved by monitoring and filtering outbound messages, ensuring the security of messaging infrastructure, receiving and acting on recipient complaints via feedback loops (FBL) from mailbox providers who offer them, and making use of third party reputation assessment and monitoring services. Glossary certification: An assessment of an identity s reputation at their behest. handling filters: A software agent that makes delivery decisions in an internet messaging system. history: The objective statistics of behavior for an identifier online. identifier: A unique online string that refers to an identity. Examples: domain name, address, IP address. identity: A real person, organization, or other accountable entity. reputation: The result of any quality assessment of an identity. reputation assessor: A person, organization, or software agent that calculates reputation. The assessor could be an internal party of either the sending or receiving message service provider or a third party. References DomainKeys Identified Mail (DKIM) Development, Deployment and Operations DomainKeys Identified Mail (DKIM) Signatures Sender ID: Authenticating Sender Reputation in a Large Webmail Service Sender Policy Framework (SPF) for Authorizing Use of Domains in MAAWG Message Sender Reputation Concepts and Common Practices 10