Electronic Mail Protocols (SMTP, POP, IMAP) It is important to (1) distinguish the user interface (i.e., your mail reader) from the underlying message transfer protocols (such as SMTP, POP or IMAP), and (2) distinguish between this transfer protocol and a companion protocol (RFC 822 and MIME) that defines the format of the messages being exchanged. Fig (1) sending and receiving emails Message Transfer For many years, the majority of email was moved from host to host using only SMTP. While SMTP continues to play a central role, it is now just one email protocol of several, Internet Message Access Protocol (IMAP) and Post Office Protocol (POP) being two other important protocols for retrieving mail messages. We ll begin our discussion by looking at SMTP and move on to POP and IMAP below. 1
To place SMTP in the right context, we need to identify the key players. First, users interact with a mail reader when they compose, search and read their email. Countless mail readers are available, just like there are many web browsers to choose from. In the early days of the Internet, users typically logged into the machine on which their mailbox resided, and the mail reader they invoked was a local application program that extracted messages from the file system. Today, of course, users remotely access their mailbox from their laptop or smartphone; they do not first log into the host that stores their mail (a mail server). A second mail transfer protocol, such as POP or IMAP, is used to remotely download email from a mail server to the user s device. Second, there is a mail daemon (or process) running on each host that holds a mailbox. You can think of this process, also called a message transfer agent (MTA), as playing the role of a post office: Users (or their mail readers) give the daemon messages they want to send to other users, the daemon uses SMTP running over TCP to transmit the message to a daemon running on another machine, and the daemon puts incoming messages into the user s mailbox (where that user s mail reader can later find them). While it is certainly possible that the MTA on a sender s machine establishes an SMTP/TCP connection to the MTA on the recipient s mail server, in many cases the mail traverses one or more mail gateways on its route from the sender s host to the receiver s host. Like the end hosts, these gateways also run a message transfer agent process. It s not an accident that these intermediate nodes are called gateways since their job is to store and forward email messages, much like an IP gateway (which we have referred to as a router) stores and forwards IP datagrams. Fig () illustrates a two-hop path from the sender to the receiver. FIG (2) Sequence of mail gateways store and forward email messages 2
Lecture 10 Independent of how many mail gateways are in the path, an independent SMTP connection is used between each host to move the message closer to the recipient. Each SMTP session involves a dialog between the two mail daemons, with one acting as the client and the other acting as the server. Multiple messages might be transferred between the two hosts during a single session. The SMTP server understands very simple text commands like HELO, MAIL, RCPT and DATA. The most common commands are: HELO - introduce yourself MAIL FROM: - specify the sender RCPT TO: - specify the recipient DATA - specify the body of the message (To, From and Subject should be the first three lines.) RSET - reset QUIT - quit the session HELP - get help on commands VRFY - verify an address Let us clarify the idea by a simple example. The following is an exchange between sending host cs.princeton.edu and receiving host cisco.com. In this case, user Bob at Princeton is trying to send mail to users Alice and Tom at Cisco. The lines sent by cs.princeton.edu are shown in black and the lines sent by cisco.com are shown in teal. Extra blank lines have been added to make the dialog more readable: HELO cs.princeton.edu 250 Hello daemon@mail.cs.princeton.edu [128.12.169.24] MAIL FROM:<Bob@cs.princeton.edu> 250 OK RCPT TO:<Alice@cisco.com> 250 OK RCPT TO:<Tom@cisco.com> 550 No such user here DATA 354 Start mail input; end with <CRLF>.<CRLF> Blah blah blah......etc. etc. etc. <CRLF>.<CRLF> 250 OK QUIT 221 Closing connection 3
As you can see, SMTP involves a sequence of exchanges between the client and the server. In each exchange, the client posts a command (e.g., HELO, MAIL, RCPT, DATA, QUIT) and the server responds with a code (e.g., 250, 550, 354, 221). The server also returns a human-readable explanation for the code (e.g., No such user here). In this particular example, the client first identifies itself to the server with the HELO command. It gives its domain name as an argument. The server verifies that this name corresponds to the IP address being used by the TCP connection; you ll notice the server states this IP address back to the client. The client then asks the server if it is willing to accept mail for two different users; the server responds by saying yes to one and no to the other. Then the client sends the message, which is terminated by a line with a single period (. ) on it. Finally, the client terminates the connection. (Note: <CRLF> stands for carriage return + line feed, which are a pair of ASCII control characters often used to indicate the end of a line of text.) Mail Reader The final step is for the user to actually retrieve his or her messages from the mailbox, read them, reply to them, and possibly save a copy for future reference. The user performs all these actions by interacting with a mail reader. This reader was originally just a program running on the same machine as the user s mailbox, in which case it could simply read and write the file that implements the mailbox. This was the common case in the prelaptop era. Today, most often the user accesses his or her mailbox from a remote machine using yet another protocol, such as POP or IMAP. The POP3 Server In the simplest implementations of POP3, the server really does maintain a collection of text files; one for each e-mail account. When a message arrives, the POP3 server simply appends it to the bottom of the recipient's file. When you check your e-mail, your e-mail client connects to the POP3 server using port 110. The POP3 server requires an account name and a password. Once you've logged in, the POP3 server opens your text file and allows you to access it. Like the SMTP server, the POP3 server understands a very simple set of text commands. Here are the most common commands: USER - enter your user ID PASS - enter your password QUIT - quit the POP3 server 4
Lecture 10 LIST - list the messages and their size RETR - retrieve a message, pass it a message number DELE - delete a message, pass it a message number Your e-mail client connects to the POP3 server and issues a series of commands to bring your e-mail messages to your local machine. Generally, it will then delete the messages from the server. You can see that the POP3 server simply acts as an interface between the e-mail client and the text file containing your messages. And again, you can see that the POP3 server is extremely simple. Fig (3) POP3 Protocol The IMAP Server The POP3 protocol is very simple. It allows you to have a collection of messages stored in a text file on the server. Your e-mail client (e.g. Outlook Express) can connect to your POP3 e-mail server and download the messages from the POP3 text file onto your PC. That is about all that you can do with POP3. Many users want to do far more than that with their e-mail, and they want their e-mail to remain on the server. The main reason for keeping your e- mail on the server is to allow users to connect from a variety of machines. With POP3, once you download your e-mail it's stuck on the machine to 5
which you downloaded it. If you want to read your e-mail both on your desktop machine and your laptop (depending on whether you're working in the office or on the road), POP3 makes life difficult. IMAP is a more advanced protocol that solves these problems. With IMAP, your mail stays on the e-mail server. You can organize your mail into folders, and all the folders live on the server as well. When you search your e-mail, the search occurs on the server machine, rather than on your machine. This approach makes it extremely easy for you to access your e-mail from any machine, and regardless of which machine you use; you have access to all of your mail in all of your folders. IMAP is similar to SMTP in many ways. It is a client/server protocol running over TCP, where the client (running on the user s desktop machine) issues commands and the mail server (running on the machine that maintains the user s mailbox) responds in kind. The exchange begins with the client authenticating him - or herself and identifying the mailbox he or she wants to access. This can be represented by the simple state transition diagram shown in Fig (4). In this diagram, LOGIN, AUTHENTICATE, SELECT, EXAMINE, CLOSE, and LOGOUT are example commands that the client can issue, while OK is one possible server response. Other common commands include FETCH, STORE, DELETE, and EXPUNGE, with the obvious meanings. Additional server responses include NO (client does not have permission to perform that operation) and BAD (command is ill formed). When the user asks to FETCH a message, the server returns it. In addition to the message itself, IMAP also defines a set of message attributes that are exchanged as part of other commands, independent of transferring the message itself. Message attributes include information like the size of the message and, more interestingly, various flags associated with the message (e.g., Seen, Answered, Deleted, and Recent). These flags are used to keep the client and server synchronized; that is, when the user deletes a message in the mail reader, the client needs to report this fact to the mail server. Later, should the user decide to expunge all deleted messages, the client issues an EXPUNGE command to the server, which knows to actually remove all earlier deleted messages from the mailbox. Finally, note that when the user replies to a message, or sends a new message, the e-mail client does not forward the message from the client to the mail server using IMAP, but it instead uses SMTP. 6
Fig (4) simple state transition diagram showing commands that the client can issue 7
The Email address You ll need an email address, of course, a sort of electronic mailbox used to send and receive messages. All such addresses follow the same approach: username@domain. These are somewhat different from web URLs, which do not use the @ (called at ) symbol. The user name: Joe_Black The user name, or user ID, identifies who is at the address in this case, Joe_Black (note the underscore). There are many ways that Joe Black s user name might be designated, with and without capital letters. Domain name: @earthlink The domain name, which is located after the @ ( at ) symbol, tells the location and type of address. Domain-name components are separated by periods (called dots ). The domain portion of the address (such as Earthlink, an internet service provider) provides specific information about the location where the message should be delivered. Top-level domain:.net The top-level domain, or domain code, is a three letter extension that describes the domain type:.net,.com,.gov,.edu,.org,.mil,.int network, commercial, government, educational, nonprofit, military, or international organization. Country:.us Some domain names also include a two-letter extension for the country.us for United States,. ca for Canada and so on. 8
Other Ways of Communicating over the Net Instant Messaging Instant messages are like a cross between email and phone, allowing communication that is far speedier than conventional email. With instant messaging (IM), any user on a given email system can send a message and have it pop up instantly on the screen of anyone else logged onto that system. (Instant messaging should not be confused with text messaging which is the exchange of messages between mobile phones nor twittering, the sending of text-based messages of up to 140 characters.) FTP for Copying All the Free Files You Want FTP (File Transfer Protocol) is a software standard for transferring files between computers with different operating systems. You can connect to a remote computer called an FTP site and transfer files to your own microcomputer s hard disk via TCP/IP over the internet. Free files offered cover nearly anything that can be stored on a computer: software, games, photos, maps, art, music, books and statistics. You can also set up your own private FTP site to enable you and other people you allow to use your site to upload and download files that are too large to send as email attachments. Some FTP files are open to the public, some are not. For instance, a university might maintain an FTP site with private files (such as lecture transcripts) available only to professors and students with assigned user names and passwords. It might also have public FTP files open to anyone with an email address. You can download FTP files using either your web browser or special software (called an FTP client program). Newsgroups for Online Typed Discussions on Specific Topics A newsgroup (or forum) is a giant electronic bulletin board on which users conducts written discussions about a specific subject. Newsgroups take place on a special network of computers called Usenet, a worldwide public network of servers that can be accessed through the internet. To participate, you need a newsreader, a program included with most browsers that allows you to access a newsgroup and read or type messages. (Messages, incidentally, are known as article). 9