Application Layer Services Omer F. Rana. Networks and Data Communications 1

Application Layer Services Omer F. Rana Networks and Data Communications 1

Application Layer Protocols Distinction between Network Application and an Application-Layer Protocol an Application-layer Protocol is only one piece of a network application. For instance: Web is a network application HTTP is an application-layer protocol. In this context, HTTP describes how messages are exchanged between the web server and the client program (Netscape, Internet Explorer etc) Also, electronic mail has many components, including mail servers that house user mailboxes, mail readers that allow users to read and create messages, a standard for defining the structure of a mail message, and application layer protocols that define how messages are exchanged between servers, servers and mail readers, and how contents of certain parts of the mail message (such as the header) are to be interpreted. Application-layer protocols in this context include the Simple Mail Transfer Protocol (SMTP), for instance (more later) Networks and Data Communications 2

Application-Layer Protocols... 2 Hence, an Application-layer protocol defines: The types of messages exchanged, for instance, request and response messages The syntax of various message types such as the fields in the message and how the fields are separated (delineated) The semantics of the fields, that is, the meaning of information contained in each field The rules for determining when and how a process sends messages and responds to these messages Such protocols can be specific to an application/vendor or may follow a standard defined in an RFC (such as HTTP) Networks and Data Communications 3

Clients and Servers An application-layer protocol generally has two parts a client side and a server side for example, a Web browser implements the client side of HTTP, and a Web server implements the server side of HTTP Clients can act as servers for other parties, and servers can act as clients for instance in a Telnet session between Hosts A and B, if Host A initiates the Telnet session (so that a user at Host A is logging onto Host B) then Host A runs the client side of the application, and Host B runs the server side On the other hand, if Host B initiates the Telnet session, then Host B runs the client side of the application In general, the initiating host is called the Client Hence, Client-Server involves the partitioning of software in an application into a set of services, which provide a set of operations to their users, and client programs which issue requests to perform a function Networks and Data Communications 4

CLIENT * MANAGES DISK STORAGE UNITS FILE SERVER * CLIENT O.S. INTERACTION APPLICATION PROGRAM OPERATING SYSTEM PROTOCOL APPLICATION PROGRAM DOES NOT TALK TO FILE SERVER DIRECTLY, BUT ONLY THROUGH O.S. CLIENT DATABASE SERVER APPLICATION PROGRAM DATABASE MANAGEMENT SYSTEM MAY DIRECTLY TALK TO DATABASE SERVER CLIENT IP Number and Port Number (131.251.42.1 80) DOMAIN NAME SERVER (DNS) APPLICATION PROGRAM SYMBOLIC NAME (www.cs.cf.ac.uk) Networks and Data Communications 5

Process Interaction An application involves two processes communicating with each other over the network refer to your operating systems notes from Year 1 The processes interact by sending and receiving messages through their sockets which can utilise the TCP or UDP protocols As discussed previously, a socket is the interface between the application layer, and the transport layer available on a host and is often referred to as the Application Layer Interface (API) between the application and the network since the socket is the programming interface with which networked applications are built Networks and Data Communications 6

Popular Applications Application: Electronic Mail Application-Layer Protocol: SMTP Underlying Transport Protocol: TCP Application: Remote Terminal Access Application-Layer Protocol: Telnet Underlying Transport Protocol: TCP Application: Web Application-Layer Protocol: HTTP Underlying Transport Protocol: TCP Application: File Transfer Application-Layer Protocol: FTP Underlying Transport Protocol: TCP Application: Remote File Transfer Application-Layer Protocol: NFS Underlying Transport Protocol: TCP or UDP Application: Streaming Multimedia Services Networks and Data Communications 7

Application-Layer Protocol: Proprietary (such as Real Networks) Underlying Transport Protocol: TCP or UDP Application: Internet Telephony Application-Layer Protocol: Proprietary (such as Vocaltec) Underlying Transport Protocol: TCP or UDP

Using Email Email has direct integration with DNS (the MX records) An Internet based mail system has three basic components: User agents responsible for helping a user construct and read emails. These are also called mail readers Mail Servers which host mailboxes a typical message will start at the sender s user agent, travel to the sender s mail server and then to the recipient s mail server finally being deposited in the recipient s mailbox Simple Mail Transfer Protocol (SMTP) an application-layer protocol based on TCP for transferring mail from the sender s mail server to the recipient s mail server Networks and Data Communications 8

RUTH JAMAL NETWORK RUTH SENDS AN EMAIL TO JAMAL RUTH JAMAL NETWORK NETWORK NETWORK NETWORK USER AGENT (UA) RELAY ONLY MESSAGE TRANSFER AGENT (MTA) LOCAL MESSAGE TRANSFER AGENT (MTA) Networks and Data Communications 9

CODES: 220: Message Transfer Agent (MTA) Available 250: Identity Accepted 251: Relaying to Recipient 354: Ready for contents Networks and Data Communications 10 450: Mailbox Busy 421: MTA Unavailable 500: Syntax Error 550: No such Recipient 552: Exceeded Storage

User User User Mail Server SMTP Mail Server User SMTP SMTP User Mail Server User Networks and Data Communications 11 User

Mail Standards SMTP used wit the TCP/IP protocol, and has been traditionally limited to the text-based electronic messages Multipurpose Internet Mail Extensions (MIME) allows various types of content (speech, images and motion video) S/MIME secure MIME RSA Data Security standard which support encrypted email transfers and digitally signed electronic mail Networks and Data Communications 12

SMTP Alice invokes her user agent for e-mail, provides Jamal s address, composes a message and instructs her user agent to send it Alice s user agent sends the message to her mail server, where it is placed in a message queue The client side of SMTP, running on Alice s mail server, sees the message in the message queue, and opens a TCP connection to an SMTP server, running on Jamal s mail server After some initial SMTP handshaking, the SMTP client sends Alice s message into the TCP connection just established At Jamal s mail server host, the server side of SMTP receives the message. Jamal s mail server then places the message in Jamal s mailbox Networks and Data Communications 13

Jamal invokes his user agent to read the message at his convenience If Jamal s mail server cannot be reached, the message is queued at Alice s server

SMTP... 2 A direct TCP connection is established, and intermediate mail servers are not normally utilised The client SMTP tries multiple times if the server SMTP is down Each email can have a list of destinations in which case the email program (user agent) makes copies of the messages and passes them onto the mail server The user may maintain a mailing list, and the email program must remove duplicates and replace mnemonic names with actual email addresses (sometimes called email alaises ) Hence, in general, an SMTP mail server processes email messages from an outgoing mail queue, and then transmits them using one or more TCP connections with the destination On transmission, the SMTP sender deletes the destination from the message list Networks and Data Communications 14

Typical Error Messages Destination host is unreachable incorrect host address, DNS not configured properly Destination host is out of operation destination host shut down Mail recipient is not available on the host TCP connection failed on the transfer of the mail a time out error occurred on the connection (perhaps due to the receiver or sender being busy, or a fault with the connection) SMTP only responsible for a message up to the point where the SMTP receiver indicates that transfer is complete it does not cater for message delivery to the user mailbox Hence, SMTP does not guarantee recovery from lost messages, and gives no end-to-end acknowledgement on successful receipt Networks and Data Communications 15

SMTP Handshaking omer@parian >mail -v omer_f_rana@hotmail.com Subject: Checking SMTP This is a message to see how SMTP works! Omer. EOT omer@parian >omer_f_rana@hotmail.com... Connecting to sentinel.cs.cf.ac.uk. via relay... 220 sentinel.cs.cf.ac.uk ESMTP Exim 3.21 #1 Mon, 05 Feb 2001 08:06:21 +0000 Networks and Data Communications 16

>>> EHLO parian.cs.cf.ac.uk 250-sentinel.cs.cf.ac.uk Hello parian.cs.cf.ac.uk [131.251.47.103] 250-SIZE 250-8BITMIME 250-PIPELINING 250 HELP >>> MAIL From:<scmofr@sentinel.cs.cf.ac.uk> SIZE=117 250 <scmofr@sentinel.cs.cf.ac.uk> is syntactically correct >>> RCPT To:<omer_f_rana@hotmail.com> 250 <omer_f_rana@hotmail.com> is syntactically correct >>> DATA 354 Enter message, ending with "." on a line by itself >>>. 250 OK id=14pgff-0004t0-00

omer_f_rana@hotmail.com... Sent (OK id=14pgff-0004t0-00) Closing connection to sentinel.cs.cf.ac.uk. >>> QUIT 221 sentinel.cs.cf.ac.uk closing connection

SMTP Handshaking... 2 The format of the transmitted message is defined in RFC 822: A header typically email addresses for sender and receiver, the time the email was sent, and any hosts involved in the transmission process Traditional: Path based addresses Unix to Unix CoPy (UUCP) sievax!mucp!unido!mcsun!cardiff!comsc!rana Alternative is the use of Domain based address Contents primarily ASCII text Networks and Data Communications 17

Mail Routing DNS defines an MX record for Mail exchangers based on domain based addresses The MTA (Message Transfer Agent) tries to deliver directly to the destination host. If this fails, it looks for a mail exchanger record, and contacts the exchanger with the lowest preference value first This could occur, for instance, if the host is on the other side of a firewall in which case the mail exchanger record (MX) would forward the mail to the firewall itself, which would then forward it over the local network Can make use of wildcards, i.e. masks, to ensure that mail is sent through mail can be routed via a mail hub, a mail server on a different network, a host-alias MX record is structured as: destination IN MX preference forwarding Networks and Data Communications 18

domain value domain For example: qatraining.com IN MX 10 smtp.qatraining.com IN MX 20 relay1.pipex.com *.qatraining.com IN MX 10 qatraining.com Hence, mail destined for addresses on the left, should be forwarded to those on the right with the lowest preference value selected first Address re-writing via MTA: pete@acme.bitnet -> pete%acme@cunyvm.cuny.edu omer@parian.cs.cf.ac.uk -> o.f.rana@cs.cf.ac.uk

omer@parian >nslookup Default Server: sentinel.cs.cf.ac.uk Address: 131.251.42.18 > set q=mx > cs.cf.ac.uk Server: sentinel.cs.cf.ac.uk Address: 131.251.42.18 cs.cf.ac.uk preference = 10, mail exchanger = stork1.cf.ac.uk cs.cf.ac.uk Networks and Data Communications 19

preference = 10, mail exchanger = crane1.cf.ac.uk cs.cf.ac.uk preference = 1, mail exchanger = sentinel.cs.cf.ac.uk cs.cf.ac.uk nameserver = pp1x.athro.cf.ac.uk cs.cf.ac.uk nameserver = ranch1x.athro.cf.ac.uk stork1.cf.ac.uk internet address = 131.251.0.32

crane1.cf.ac.uk internet address = 131.251.0.31 sentinel.cs.cf.ac.uk internet address = 131.251.42.18 pp1x.athro.cf.ac.uk internet address = 131.251.0.4 ranch1x.athro.cf.ac.uk internet address = 131.251.0.10

From java-for-cse-owner@postoffice.npac.syr.edu Fri Nov 21 17:34:20 1997 Return-Path: <java-for-cse-owner@postoffice.npac.syr.edu> Delivery-Date: Fri, 21 Nov 1997 17:34:20 +0000 Received: from postoffice.npac.syr.edu by sentinel.cs.cf.ac.uk with SMTPX (PP); Fri, 21 Nov 1997 17:30:56 +0000 Received: (from majordom@localhost) by postoffice.npac.syr.edu (8.7.5/8.7.1) id IAA14029 for java-for-cse-outgoing; Fri, 21 Nov 1997 08:59:27-0500 (EST) X-Authentication-Warning: postoffice.npac.syr.edu: majordom set sender to owner-java-for-cse using -f Networks and Data Communications 20

Received: from milan.doe.ernet.in (milan.doe.ernet.in [202.41.99.2]) by postoffice.npac.syr.edu (8.7.5/8.7.1) with SMTP id IAA14024 for <java-for-cse@npac.syr.edu>; Fri, 21 Nov 1997 08:59:22-0500 (EST) Received: from cdacb.ernet.in by milan.doe.ernet.in (4.1/SMI-4.1) id AA11191; Fri, 21 Nov 97 19:19:25+050 Received: by cdacb.ernet.in (5.x/SMI-SVR4) id AA23023; Fri, 21 Nov 1997 19:22:41-0500 Date: Fri, 21 Nov 1997 19:22:41-0500 From: raj@cdacb.ernet.in (RajKumar)

Debugging Mail Servers omer@parian >telnet sentinel 25 Trying 131.251.42.18... Connected to sentinel.cs.cf.ac.uk. Escape character is ^]. 220 sentinel.cs.cf.ac.uk ESMTP Exim 3.21 #1 Mon, 05 Feb 2001 08:12:15 +0000. 500 Unrecognized command QUIT 221 sentinel.cs.cf.ac.uk closing connection Connection closed by foreign host. POP servers are on port 110, and IMAP servers on port 143 Use the ping command to see if host is still available Check system log files Check the Received: lines of messages and identify both the time of transmission of the last message, and any intermediate relay agents involved Networks and Data Communications 21

MIME SMTP is insufficient: Can only transmit text messages not executable files or other binary objects Does not allow attachment of files and can only use a 7-bit ASCII character set (no support for extended ASCII cannot use symbols!) Multipurpose Internet Mail Extensions (MIME) RFC 1521/1522 which adds: New message header fields such as Content-type, Content-transfer-encoding, Content-id, Content-Description, MIME-version Support for various multimedia content formats Transfer encodings for transforming attached files Networks and Data Communications 22

MIME Content Types Content types define the format of the attached files there are 16 different content types in seven different content groups A MIME message can contain multiple attachments, in which case, the content-type header field includes a boundary which acts as a delimiter between the multiple attachments MIME content types include: text/plain, text/richtext (such as HTML), multipart/mixed (independent attachments, order important), multipart/parallel (independent attachments, order not important), multipart/alternative (attachment is a different version of original data), multipart/digest (similar to multipart/mixed but uses rfc822 (SMTP) to define each message) Networks and Data Communications 23

message/rfc822, message/partial (to fragment a mail message), message/external-body (define a pointer to an external object) image/jpeg, image/gif video/mpeg, video/mov, audio/au, audio/wav, audio/basic application/postscript, application/octet-stream

Mail Access Protocols Assumed, so far, that mail reader is run on the mail server (standard until 1990) however, today, mail reader is on local host Aim to make the mail server (which receives and queues messages) independent of the mail reading program. The mail server is always connected to the Internet, whereas a mail reader can be invoked when required. Mail servers generally maintained by ISPs Protocol to communicate between mail reader and mail server necessary (as they are now on different hosts) This protocol is used to transfer messages from Jamal s mail server to his local PC POP3 (Post Office Protocol version 3) and IMAP (Internet Mail Access Protocol) SMTP is a push based protocol, whereas Mail Access Protocols are based on pull (like HTTP) Networks and Data Communications 24

User Agent SMTP SMTP POP3 or IMAP User Agent Alice Sender s mail server Recipient s mail server Networks and Data Communications 25 Jamal

Post Office Protocol Extremely simple and enables many different programs to make use of it Aim to create a standard method for users to access a mail server. Begins with a user agent opening a TCP connection to the mail server on port 110. Once the connection has been established, POP3 goes through three phases: 1. Authorisation: user name and password are sent to authenticate the user Networks and Data Communications 26

2. Transaction: the user agent retrieves the messages. The user agent can mark messages for deletion and obtain mail statistics. The mailbox is locked during this process 3. Update: occurs after the client has issued the quit command, ending the POP3 session at this time the mail server deletes the marked messages

Extended POP POP is a problem with nomadic users for instance, if Bob wants to access his mail from multiple machines, including the office PC, the home PC, and a portable computer, the download-and-delete operation in POP will scatter Bob s email across all of his machines In particular, if Bob first reads a message from his home PC, he will not be able to read the message on his laptop, although he is being connected to the same mail server Alternately, in the download-and-keep mode, the user agent leaves the messages on the mail server after downloading them Although a POP3 server maintains some state information about a session between a mail server and a user agent (such as which messages have been deleted), it does not carry state information across POP3 sessions Networks and Data Communications 27

IMAP (Internet Mail Access Protocol) POP3 has limited functionality for instance, it is not possible to remotely manage mail archives on the mail server with POP this is a problem for nomadic users IMAP has more features that POP, but also significantly more complex see RFC 2060 Designed to enable users to manipulate remote mailboxes as if they were local such as archiving mail in folders, search remote folders for messages matching a criteria etc Enables a user agent to obtain components of a message such as the headers only, or just one part of a multipart MIME message (useful for low bandwidth, wireless connections) The transaction state of the POP3 server is replaced by a selected state on an IMAP server. In this state, a user can issue a host of commands to the mail server. The other states are more or less equivalent to POP3 Networks and Data Communications 28

IMAP is supported in Unix and WinNT/2000 servers also supported within third party applications such as Lotus Notes and Microsoft Exchange. Clients can include Netscape Communicator and Eudora Pro An Application Configuration Access Protocol is being defined to enable IMAP based servers to support additional services, such as an address book

Continuous-media Email Email that includes audio or video in particular resembles ordinary telephone voice-mail messaging in some respects (but much more powerful) Differs from ordinary email, in that messages can be much larger, there is a more stringent end-to-end delay requirement, and has greater sensitivity to recipients with highly heterogeneous Internet access rates and local storage capabilities Current mail access protocols (such as POP3, IMAP and HTTP) are not adequate for CM email Very much the future enable email based on streaming protocols Networks and Data Communications 29

Email Security Most email is transmitted in plaintext over the network also SMTP conversations are unauthenticated, and POP communications are authenticated using only a password (also in plaintext) No official standard for encryption of data on the Internet primarily three levels of concern 1. Authentication ensure that both parties are who they claim to be 2. Confidentiality prevent unauthorised viewing of data 3. Integrity ensure that data is not altered Pretty Good Privacy (PGP) most widely used. Uses RSA public key algorithm to distributed encryption keys nd authenticate messages. A user encrypts a message and then mails the output. Some mail clients will automatically decrypt it if they have key for the user. Users must build up Networks and Data Communications 30

key rings, containing the public keys of the users they wish to communicate with Privacy Enhanced Mail (PEM) is a commercial secure mail system and supports authentication based on certificates. Secure/MIME (S/MIME) introduces extra headers to specify an encryption mechanism and authentication

Designing a Mail Network Choose the following components: Mail Host central point of mail routing individual machines that do not know where to send a mail message should, by default, send it here. Must know about all contactable machines and domains Mail Relay forwards mail intended for destinations outside the organisation. May also receive incoming email. Relay talks to peers through the same mail language (such as SMTP) Mail Gateway forms a link between two differing mail systems, and also forwards mail outside the organisation Mail Server stores mailboxes for clients. Incoming messages are stored here until the client retrieves them (POP3) Mail Client These are functional components, and not individual machines. Networks and Data Communications 31

Organisation Boundary Internet External Mail domains Mail Relay AOL Mail Gateway Mail Host Mail Server Mail Server Mail Client Mail Client Mail Domain Mail Client Mail Client Networks and Data Communications 32

JavaMail JavaMail is an object-oriented API to email messaging systems. JavaMail, the API, is strictly protocol-independent, and it is up to service providers to implement specific protocols (very much like JDBC for databases) Can be used directly (or with intermediate software) with SMTP, POP3 and IMAP Networks and Data Communications 33

First, obtain a reference to a Session object through Session.getDefaultSession(). Sessions manage configuration options and authentication. import java.io.*; import java.util.*; import javax.mail.*; import javax.mail.event.*; import javax.mail.internet.*; class FunnyMail { private Session session; private URLName urlname; Networks and Data Communications 34

public FunnyMail(String url) { session = Session.getDefaultInstance(System.getProperties(), null); urlname = new URLName(url); } A URL is of the form: protocol://username:password/hostname, pop3://fun:pwd@mail.psol.com imap://javamail:pwd@imap.psol.com

for a fun mailbox on mail.psol.com, or a javamail mailbox on an IMAP server Accessing the mailbox: private void readandreply() throws MessagingException, IOException { Store store = session.getstore(urlname); store.connect(); Folder inbox = store.getfolder("inbox"); inbox.open(folder.read_write); int count = inbox.getmessagecount(); for(int i = 1;i <= count;i++) {

Message m = inbox.getmessage(i); replyto(m); m.setflag(flags.flag.deleted,true); } inbox.expunge(); inbox.close(true); store.close(); } private void replyto(message m) throws MessagingException, IOException { System.out.print("replying to: "); System.out.println(m.getSubject()); Message r = new MimeMessage(session);

r.setfrom(internetaddress.getlocaladdress(session)); r.setrecipients(message.recipienttype.to,m.getfrom()); r.setsubject("fun: " + m.getsubject()); r.setsentdate(new Date()); r.setheader("x-mailer","funnymail 1.0"); Object o = m.getcontent(); if(m.ismimetype("text/plain") && o instanceof String) r.settext(reverse((string)o)); else r.settext("funnymail requires plain text mails!"); Transport.send(r); }