Mail Protocol, Postfix and Mail security

Transcription

1 Mail Protocol, Postfix and Mail security

2 How Appears to Works

3 How Really Works

4 Message Format Envelop Routing information for the "postman" Message Header Sender Recipients (simple, lists, copies, blind copies) Other fields of control (date, subject) Message Body Free text Structured document (i.e.: MIME)

5 Message Format From: GZ Kabir To: Diep Kong < > Cc: Moin < > Subject: How Internet mail works Hi Kong, I'm going to be running a course on... ü Format was originally defined by RFC 822 in 1982 ü Now superseded by RFC 2822 ü Message consists of ü Header lines ü A blank line ü Body lines

6 Message Format Embedded MUA uses inter-process call to send to MTA Freestanding MUA uses SMTP to send mail Headers added by the MUA before sending From: GZ Kabir To: Diep Kong < > Cc: Moin < > Subject: How Internet mail works Hi Kong, I'm going to be running a course on...

7 Headers added by MTAs Mail Delivery Agent (MDA) / Mail Transfer Agent (MTA) From: GZ Kabir <gzkabir@office.bdcom.com> To: Diep Kong < diepkong@gmail.com > Cc: Moin < moin@bofh.im > Subject: How Internet mail works..

8 A message in Transit A message is transmitted with an envelope: MAIL FROM:<gzkabir@office.bdcom.com> RCPT TO:<moin@bofh.im> The envelope is separate from the RFC 2822 message Envelope (RFC 2821) fields need not be the same as the header (RFC 2822) fields MTAs are (mainly) concerned with envelopes Just like the Post Office... Error ( bounce ) messages have null senders or Postmaster s MAIL FROM:<>

9 An SMTP Session telnet mail7i.protonmail.ch mail7i.protonmail.ch ESMTP Postfix... EHLO mail-pg0-f54.google.com 250- mail7i.protonmail.ch SIZE PIPELINING 250 HELP MAIL 250 OK RCPT 250 Accepted DATA 354 Enter message, ending with. (continued >>>>) Received: from... From:... To:... etc OK id=10spdr-00034h-00 quit 221 mail-pg0-f54.google.com closing conn... SMTP return codes 2xx OK 3xx send more data 4xx temporary failure 5xx permanent failure

10 DNS Resolution and Transfer Process To find the recipient's IP address and mailbox, the MTA must drill down through the DNS system, which consists of a set of servers distributed across the Internet beginning with the root name servers root servers refer requests for a given domain to the root nameservers that handle requests for that tld MTA can bypass this step because it has already knows which domain nameservers handle requests for these.tlds e.g. bofh.im asks the appropriate DNS server which Mail Exchange (MX) servers have knowledge of the subdomain or local host in the address DNS server responds with an MX record: a prioritized list of MX servers for this domain To the DNS server, the server that accepts messages is an MX server. When is transferring messages, it is called an MTA. MTA contacts the MX servers on the MX record in order of priority until it finds the designated host for that address domain sending MTA asks if the host accepts messages for the recipient's username at that domain (i.e., username@domain.tld) and transfers the message

11 Firewalls, Spam and Virus Filters An encountering a firewall may be tested by spam and virus filters before it is allowed to pass inside the firewall filters test to see if the message qualifies as spam or malware If the message contains malware, the file is usually quarantined and the sender is notified If the message is identified as spam, it will probably be deleted without notifying the sender.

12 Troubleshooting Issues transient failures If a transient error occurs, the MTA will hang onto the message, periodically retrying the delivery until it either succeeds or fails, or until the MTA decides that the transient issue is really a permanent condition. permanent failures If the MTA cannot deliver the message (it has received a fatal error message or failed to complete the transfer after repeated attempts), it bounces the message back to the sender. If the sender is a mailing list, the bounce may be handled by automated bounce-handling software.

13

14 Concept of Mail Protocols

15 Component of system Mail Transport Agent/Message Transfer Agent (MTA) Mail User Agent (MUA) Mail Delivery Agent(MDA)

16 MTA The actual mail transfer is done through message transfer agents (MTAs). To send mail, a system must have the client MTA, and to receive mail, a system must have a server MTA. The formal protocol that defines the MTA client and server in the Internet is called Simple Mail Transfer Protocol (SMTP) SMTP uses commands and responses to transfer mail between an MTA client and MTA server.

17 SMTP Mail transfer Flow

18 MTA connection setup

19 MTA Connection Setup..Contd.. Sender opens TCP connection with receiver Once connected, receiver identifies itself 220 service ready Sender identifies itself HELO <domain> Receiver accepts sender s identification 250OK If mail service not available, step 2 above becomes: 421 service not available

20 SMTP Exchange of command response pair

21 Message Transfer Sender may send one or more messages to receiver Each message transfer has the following phases: One MAIL command, identifies originator Gives reverse path to use for error reporting Receiver returns 250 OK or appropriate fail/error message One or more RCPT commands, identifies recipients for the message Each recipient identified by a separate RCPT Separate reply for each recipient (250 OK etc.) One DATA command transfers message text End of message indicated by line containing just period (.)

22 MTA connection termination

23 MTA connection termination Sender sends QUIT and waits for reply Then initiate TCP close operation Receiver initiates TCP close after sending reply to QUIT

24 Optimization If message is sent to multiple users on a given host,it is sent only once. Delivery to users handled by destination host If Multiple messages are ready for given host,a single TCP connection can be used. Saves overhead of setting up and termination of connection

25 Possible Errors Host unreachable Host out of operation TCP connection fail during transfer Faulty destination address User error Target user address has changed Redirect if possible Inform user if not Sender re-queue the mail - will retry till a configurable period of time

26 SMTP protocol reliability TCP connection is used to transfer mail from sender to receiver over TCP connection Attempts to provide reliable service No guarantee to recover lost messages No end-to-end ack to sender Error indication report not guaranteed

27 SMTP receiver Accepts arriving message Places in user mailbox or copies to outbound message queue for forwarding Receiver must verify local mail destination deal with errors Transmission Lack of disk space

28 SMTP status codes(dsn- Delivery status Notification) Leading digits Indicates catagories 2XX-Positive Completion Reply(Successful) 3XX-Positive Intermediate Reply(Redirection) 4XX-Transient negative completion reply(client error) 5XX-Permanent negative completion reply(server error)

29 SMTP status codes..cont.

30 Problems with SMTP No inherent security n Authentication n Encryption Only uses NVT (Network Virtual Terminal) 7- bit ASCII format

31 s can be forged.. HELO mail.rose.edu MAIL FROM: RCPT TO: DATA From: Dr. Art Zenner To: Professor Richards Subject: CIT 2243 Professor Richards, By department decree all students in your Introduction to Unix class are hereby to be granted automatic A s. Thank you, Dr. Art Zenner. QUIT

32 Extensions to SMTP MIME Multipurpose Internet Mail Extensions n Transforms non-ascii data to NVT (Network Virtual Terminal) ASCII data w w w w w Text Application Image Audio Video

33 MIME and Base64 Encoding If the internet is the information highway, then the path for is a narrow tunnel n Only very small vehicles can pass trough Then how do you send a big truck through a small ravine? n You have to break it down to smaller pieces and transport the pieces through the ravine, and reassemble the truck

34 MIME and Base64 Encoding The same happens when you send a file attachment via . This is known as encoding n the binary data (256 bits) is transformed to ASCII text (128 bits n allowing it to fit through the tunnel On the recipient's end, the data is decoded and the original file is rebuilt.

35 Mail Transfer Agents (MTA) MTAs do the actual mail transfers MTAs are not meant to be directly accessed by users. Other MTA s are: n n n n n n Postfix Qmail MS Exchange CC:Mail Lotus Notes.etc.

36 Problems with simple SMTP The first one relates to message length. Some older implementations cannot handle messages exceeding 64KB. Another problem relates to timeouts. If the Client and server have different timeouts, one of them may give up while the other is still busy, unexpectedly terminating the connection. Infinite mail storms can be triggered. For example, If host 1 holds mailing list A and host 2 holds mailing list B and each list contains an entry for the other one, then a message sent to either list could generate a never ending amount of traffic unless somebody checks for it.

37 ESMTP (RFC 2821) To get around the problems with simple SMTP, extended SMTP has been defined in RFC Clients wanting to use it should send an EHLO message instead of HELO initially. If this is rejected, then the server is a regular SMTP server, and the client should proceed in the usual way. If the EHLO is accepted, then new commands and parameters are allowed.

38 POP3 Mail access protocol: POST OFFICE PROTOCOL [RFC 1939] Simple and limited functionality,consists of client software and Server software, Server performs user authorization IMAP: Internet mail access protocol [Version 4]

39 POP3 Retrieves messages from a mail server Typically, messages are downloaded to your mail client, and deleted from the server Designed for use with dial-up connections when people were intermittently connected Listens on Port 110 (with Secure POP generally on port 995)

40 POP3 Connection Establishment

41 POP3 protocol session ~]# telnet mail.amberit.com.bd 110 Trying Connected to mail.amberit.com.bd. Escape character is '^]'. +OK Dovecot ready. user +OK pass letmein +OK Logged in. list +OK 1 messages: retr 1 +OK 482 octets <snip>message Headers</snip> This is my short message quit +OK Logging out.

42 Basic POP3 commands USER <name> - identifies the user PASS <password> - authentication for user STAT - lists all messages in the mailbox of user LIST <msg no.> - lists the content of a message RETR <msg no.> - retrieves a particular message DELE <msg no.> - Deletes a particular message NOOP RSET QUIT Replies +OK ERR

43 IMAP Protocol session ~]# telnet mail.amberit.com.bd 143 Trying Connected to mail.amberit.com.bd Escape character is '^]'. * OK Dovecot ready. A1 LOGIN user@testing.comletmein A1 OK Logged in. A2 SELECTInbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) *

44 IMAP Protocol session..cont. OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 0 RECENT * OK [UIDVALIDITY ] UIDs valid * OK [UIDNEXT 2] Predicted next UID A2 OK [READ-WRITE] Select completed. A3 FETCH 1 BODY[HEADER] * 1 FETCH (BODY[HEADER] {454} <snip> Message Header Delivered</snip> A3 OK Fetch completed. A4 LOGOUT * BYE Logging out A4 OK Logout completed. Connection closed by foreign host.

45 Webmail

46 Webmail Webmail (or web-based ) is any client implemented as a web application running on a web server Online in nature IMAP connections mostly Secured HTTP (HTTPS) must Can be hosted, e.g.: RoundCube, SquirrelMail, Horde, Zimbra, Rainloop, Kite, MailPile,iRedMail, etc. Webmail Service providers : Gmail, Outlook, AOL, Yahoo, etc. Privacy and Security Concern Easy to configure, easy to host.

47 Common Threat Landscape Eavesdropping Spamming and Phishing Spoofing Malicious Attachments Replying and Forwarding Issues CC & BCC Issues

48

49 Postfix

50 Short History Originally developed in the late 90s at IBM by Wietse Venema, author of security software (SATAN, TCPwrappers,...), as IBM Secure Mailer Place under an Open Source license, and renamed Postfix Intended as a replacement for then insecure mail systems, such as Sendmail

51 Design Goals Safety Robustness Performance Modularity Compatibility

52 Safety Postfix makes it very hard to lose mails many checks to ensure that mail has been written to disk or delivered Back off mechanisms in case of repeated failure

53 Security Collection of daemons working together Doesn't use environment for communication Very paranoid about input checking, all allocation is dynamic (avoiding buffer overflows) chroot support out of the box for almost all processes & daemons No data is ever exchanged directly between processes all is done via IPC, and files on disk Conservative resource usage

54 Performance Designed to be fast from the ground up Also behaves well with neighbors, doesn't flood them with mail, and instead uses a throughput adaptation Will not block delivery for a message if one recipient domain fails

55 Modular One program, one function All programs controlled from master.cf Many small programs working together, with limited privileges Compatible with Sendmail's /etc/aliases and.forward conventions

56 Features Virtual domains domains and users are completely independent of system (UNIX) users Aliases sendmail compatible Rewriting senders, recipients, globally RBL support (Realtime Blackhole Lists) support Content filtering using pipes, SMTP or milter Support for arbitrary mail manipulation with policy services (custom programs talking to postfix)

57 More Features Restriction classes Conditional filtering Sender or recipient address verification (test addresses before accepting mail from them) TLS support

58 Core Concepts : maps In postfix, everything is looked up in a map (table) Maps can be in many formats or use many data sources: hash/btree regexp/pcre CIDR NIS LDAP, *SQL (user defined queries)

59 Architecture Architecture

60 Basic Postfix Configuration Two primary configuration files main.cf Main configuration file where all the subsystems are configured (smtp, smtpd, cleanup, routing,...) master.cf File controlling how the master process of postfix will launch all the necessary postfix daemons to perform mail routing, on-demand

61 Other Configuration Details Reside in maps mentioned earlier Tables containing values and conditions, referred to from main.cf, controlling all aspects such as: Virtual and local domains Routing rules Access control Rewriting...

62 Configuration: postconf command postconf used to view and edit configuration parameters For changing the configuration, it is usually done vi editing main.cf directly

63 Some Basic main.cf smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination myhostname = group1.group01.net alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = /etc/mailname mydestination = group1.group01.net, localhost.group01.net,, localhost relayhost = group1.group01.net mynetworks = / /24 [2001:df0:a:4::]/64 [::ffff: ]/104 [::1]/128 mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all inet_protocols = all

64 # TLS parameters Some Basic main.cf smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

65 ## in the file /etc/mailname Some Basic main.cf group1.group01.net ## in the file /etc/aliases postmaster: root sysadm: apnic

66