Internet Applications. Review BUPT/QMUL

Transcription

1 Internet Applications Review BUPT/QMUL

2 Network Basics (1) What is the Internet? Internet vs. internet Internet vs. www How do machines communicate with one another on the Internet? What are the major components of Internet? Internet applications Internet protocols Internet addresses Physical infrastructure What is protocol? Protocol = A set of rules for communicating HTTP/FTP/SMTP, TCP/UDP, IP 2

3 Q2: How does the Internet work? communication on the Internet The source computer One application produces the data to send The software packetize the data Local Area Network Ruby A The cables connecting the computers to the network The network device receives the data and pushes them out The software chooses the path for data delivery The destination computer The software depacketize the data The counterpart to the sender application reads the data B 3

4 Q2: How does the Internet work? communication on the Internet Application (e.g., IE) Application (e.g., Web Server) Data Network Software (i.e. TCP/IP) Data Network Software (i.e. TCP/IP) Packet Packet Packet Packet Packet Packet Physical (Hosts, Routers, Wiring) 4

5 Network Basics (2) What are the two important design concepts of Internet? Layered networking model Client-server paradigm Terms for network devices and examples node, host node, link, network component Terms for network performance parameters: bandwidth, delay, jitter, error rate Network types according to the switching function in the network Circuit switching network Message switching network Packet switching network Hybrid switching network 5

6 Network Basics (3) Different channel access technologies multi-access and point-to-point Network types according to the range of the network LAN, MAN, WAN, PAN Network types according to the user of the network public network and private network Layered architecture ISO/OSI model, TCP/IP model Revisory Model Devices at different layers Hub, Bridge/Switch, Router, Gateway 6

7 Network Programming Basics (1) Basic concepts Process: what is a process? Is process equals to program? PID, PPID Special processes: init(pid=1) fork(), exec() System call: what is a file descriptor? Special file descriptors: 0,1,2 Related system calls and their functions: open(), read(), write(), lseek() Signal: what is a signal? Which conditions will generate signals?: 5 conditions What can a process do with a signal? : 3 choices Signal() system call and its functions 7

8 Network Programming Basics (2) IP Addresses IP address structure Big-endian and little-endian, HBO and NBO byte order conversion functions: htonl(), ntohl(), htons(), ntohs() DNS Host entry structure System calls for retrieving host entries gethostbyname( ), gethostbyaddr( ) Connection What is a connection? How to identify an endpoint of a connection? How to identify a connection? Client-IP, client-port, server-ip, server-port Well-known port numbers for typical applications DHCP, DNS, TELNET, TFTP, FTP, SMTP, POP3, HTTP, SNMP 8

9 Network Programming Basics (3) Functions of netstat, ifconfig and ping Sockets interface What is a socket? What is the sockets descriptor? Internet-specific socket address and Generic socket address struct sockaddr vs. struct sockaddr_in how do we use them in the system calls such as connect(), bind(), and accept()? Different sockets interface types SOCK_STREAM, SOCK_DGAM, SOCK_RAW System calls used in sockets programming Socket operation: socket(), bind(), send(),recv(), close() Byte order operation: htonl(), Address formats conversion: inet_aton(), inet_ntoa(), Name and address operation: gethostbyname(), Basic flows of TCP/UDP based sockets API 9

10 Overview of TCP-based sockets API TCP Client socket() TCP Server socket() bind() listen() connect() send() recv() close() Connection request data data EOF accept() recv() send() read() Await connection request from next client close() 10

11 Overview of UDP-based sockets API UDP Client socket() UDP Server socket() bind() sendto() data recvfrom() recvfrom() data sendto() close() 11

12 DHCP DHCP overview Basic function: automatic configuration of remote hosts relationship between BOOTP and DHCP DHCP client(port 68), DHCP server(port 67) DHCP Relay: forward DHCP messages between subnets. DHCP lease: amount of time that the DHCP server grants to the DHCP client Phases of IP assignment with DHCP DHCP message format STD and Message Sequence Chart of DHCP Address acquisition Procedure of Early lease termination in DHCP Procedure of Lease renewal in DHCP Basic address acquisition procedure through DHCP relay 12

13 Phases of IP Assignment with DHCP the client makes a request for an IP address the DHCP servers offer an IP address the client selects an offer and requests an IP lease the selected DHCP server assigns an IP to the client 13

14 DHCP Message Format OP (1) HTYPE (1) HLEN (1) HOPS (1) TRANSACTION ID (4) SECONDS (2) FLAGS (2) CLIENT IP ADDRESS (4) YOUR IP ADDRESS (4) SERVER IP ADDRESS (4) ROUTER IP ADDRESS (4) CLIENT HARDWARE ADDRESS (16) SERVER HOST NAME (64) BOOT FILE NAME (128) OPTIONS (variable)

15 Address Acquisition: MSC Server (not selected) Client Server (selected) Begin initialization DHCPDISCOVER Determines configuration DHCPOFFER DHCPDISCOVER DHCPOFFER Determines configuration DHCPREQUEST Collects replies & selects configuration DHCPREQUEST DHCPACK 15

16 Address Acquisition: STD of the client Select Offer/ DHCPREQUEST Host Boots /DHCPDISCOVER SELECT DHCPOFFER INIT State Transition of DHCP Client REQUEST DHCPACK BOUND 16

17 Early Lease Termination: Procedure Client Server (selected) Graceful shutdown DHCPRELEASE Discards lease 17

18 Lease Renewal: Procedure (1) T1 expires Client Server (selected) Client Server (selected) T1 expires T1 expires DHCPREQUEST DHCPREQUEST DHCPACK DHCPNAK Reset T1 and T2 T1: the time at which the client enters the RENEW state and attempts to contact the server that originally issued the client's network address 0.5 * duration_of_lease T2: the time at which the client enters the REBIND state and attempts to contact any server * duration_of_lease

19 Lease Renewal: Procedure (2) Both T1 and T2 expire, address rebound Server (others) Client T1 expires Server (selected) DHCPREQUEST T2 expires DHCPREQUEST DHCPREQUEST DHCPACK Reset T1 and T2 19

20 DNS(1) Basic functions of DNS: converting between hostname and IP address Current status: hierarchical structure, distributed database, general-purpose Hierarchical structure of domain namespace Important terms Domain / domain name / FQDN Domain namespace, zone Resource Record Name server / primary server / secondary server / caching server Resolver Query / response Standard query / inverse query / pointer query recursive resolution / iterative resolution 20

21 Hierarchical structure Unnamed root Top level domains arpa com edu gov int mil net org ae cn zw 2nd level domains in-addr mit edu 59 cs bupt 64 xx xx.cs.mit.edu cs.bupt.edu.cn cs Generic domains in-addr.arpa Country domains 21

22 DNS(2) How does DNS work together with the user programs (e.g. TELNET, FTP, HTTP, SMTP)? Procedure of the recursive resolution and iterative resolution What are the mechanisms in DNS that are possible to improve the querying efficiency? The ideas of inverse query and pointer query. The comparison between them. DNS Message Format Types of Resource Record (only the one highlighted using red color in the lecture notes) A, NS, MX, CNAME, PTR 22

23 Communication model between user program, resolver and name server Local Host Foreign User Program User queries User responses Resolver Queries Responses DNS name server Cache additions references cache Resolvers are normally implemented in system calls. E.g. gethostbyname(), gethostbyaddr() 23

24 Recursive resolution 1 6 Local Name Server

25 Iterative resolution 25

26 DNS Message Format(from RFC 1035) Query and Response messages, both with same message format Parameter(Flags) ID QR OPCODE AA TC RD RA Z Rcode 31 Question count Authority count HeaderAnswer count Additional count Question Section (variable number of questions) Answer Section (variable number of RRs) Authority Section (variable number of RRs) Additional Section (variable number of RRs) 26

27 Telnet(1) What is TELNET and telnet? TELNET: a protocol used to establish a dumb terminal session to another computer on the Internet telnet: a program that supports the TELNET protocol over TCP What are the advantages of the idea of option negotiation in TELNET? NVT What is NVT? What are its functions? Network Virtual Terminal, providing a standard interface to remote systems NVT operations Understand data path from the user s keyboard to the remote system 27

28 NVT Operation User s keyboard & display Accommodating heterogeneity Converting client system format into NVT format TELNET Client TCP connection across Internet TELNET Server Converting NVT format into server system format Server s system Converting NVT format into client system format Client System format Server System format NVT format Converting server system format into NVT format TELNET client and server convert between native format and NVT format 28

29 Transmission Of Data Data path from the user s keyboard to the remote system Client reads from terminal TELNET client Client sends to server (NVT) TELNET server User s keyboard & display Operating system Server receives from client (NVT) Operating system Server sends to pseudo terminal TCP/IP internet 29

30 Telnet commands TELNET control functions: IAC, DO, DONT, WILL, WONT TELNET options example Echo modes Binary transmission Line mode vs. character mode Character set Terminal type Understand the TELNET session through examples 30

31 Telenet Control Functions DO, DONT, WILL, WONT Used for options negotiation Examples Sender Receiver Meaning WILL DO WILL DON T DO WILL DO WONT Sender wants to active a option, and receiver agrees Sender wants to active a option, and receiver refuses Sender wants receiver to active a option, and receiver agrees Sender wants receiver to active a option, and receiver refuses 31

32 Example: Negotiation of Echo Option Client Server Do enable the echo option ECHO DO IAC IAC WILL ECHO I will enable the echo option 32

33 Other Remote Access Technologies Remote login in text-based system telnet SSH Rlogin 33

34 TFTP/FTP(1) TFTP features Read and write files from / to remote computers Minimal overhead TFTP packet format (you can ignore the errcode definition) Read request, Write request, Data, Acknowledgment, Error Transfer mode of TFTP: Netascii, Octect Typical communication procedure of TFTP Retransmission defined in original TFTP protocol The sorcerer s apprentice syndrome problem and how to fix it 34

35 TFTP Protocols Packet Format Ethernet Frame Header IP Header UDP Header TFTP Header Read request (RRQ) 01 filename 0 mode 0 2 byte opcode in network byte order null terminated ASCII string containing name of file null terminated ASCII string containing transfer mode Variable length fields 35

36 TFTP Example Beginning with RRQ/WRQ Every good block is Acknowledged Both sides use timers Simple automatic repeat request mechanism provides flow control A block of less than 512 bytes signals the end of the file timeout resend ascii ACK 1 Lost ACK 2 ACK file1 DATA DATA DATA DATA RRQ Block1 512 bytes Block2 512 bytes Block2 512 bytes Block3 512 bytes 3 Lost ACK 3 Damaged ACK 4 client DATA DATA 4 server Block4 464 bytes 4 Block4 464 bytes 36 timeout resend timeout resend

37 TFTP Operations Retransmission Symmetric Both machines involved in a transfer are considered senders and receivers. One sends data and receives acknowledgments The other sends acknowledgments and receives data Each side implement the timeout and retransmission If a data packet gets lost in the network, the data sender times out and retransmits the last data packet If an acknowledgment is lost, the acknowledgment sender retransmits the last acknowledgment The sender has to keep just one packet on hand for retransmission, since the lock step acknowledgment guarantees that all older packets have been received Duplicate data packets must be recognized (ignored) and acknowledgment retransmitted This original protocol suffers from the sorcerer s apprentice syndrome (SAS) 37

38 TFTP Operations Sorcerer s Apprentice Syndrome Send DATA[n] timeout Retransmit DATA[n] Sender Receive ACK[n] Send DATA[n+1] Receive ACK[n] (dup) Send DATA[n+1] (dup) and so on Receiver Receive DATA[n] Send ACK[n] Receive DATA[n] (dup) Send ACK[n] (dup) Receive DATA[n+1] Send ACK[n+1] Receive DATA[n+1] (dup) Send ACK[n+1] (dup) Arising when an acknowledgment for a data packet is delayed, but not lost Leading to excessive retransmissions Once started, the cycle continues indefinitely with each data packet being transmitted exactly twice 38

39 TFTP/FTP(2) FTP features Used to transfer files between hosts Used to manipulate files, FTP model FTP basic control commands and replies USER, PASS, CWD, CDUP, QUIT, PORT, PASV LIST, RETR, LIST, FTP Control Connection & Data Connection FTP Active and Passive Mode Anonymous FTP log in with anonymous as the account name Give your address as password Only for downloading 39

40 FTP Model User Interface User Server PI Control Connection Client PI File System Server DTP Data Connection Client DTP File System FTP server FTP client 40

41 Active and Passive mode Typical data connection handling sequence (in active mode) Client sets up to listen on a unique port Client uses local socket information to send PORT command to server Server responds with 200 reply to acknowledge the port number Client sends RETR, STOR, or other transfer command Server sends preliminary reply Server does active open ( connect ) File data sent over connection Server sends 226 or other reply Server/client closes data connection Another mode: passive mode Client sends command PASV server listens to a specific port and client should access that port 41

42 FTP Control Connection & Data Connection (1) Server 20 data 21 control Client 2189 control 5001 data Active Mode: Server port for data connection =20 Control connection is initiated by client and established PORT (5001) 200 Control command (RETR) Request to establish the data connection Acknowledgment to the request Reply (150) data Reply (226) Receive user command Acquire port

43 FTP Control Connection & Data Connection (3) Server 6077 data 21 control Client 2189 control 5001 data Passive Mode: Server port for data connection >1024 Control connection is initiated by client and established PASV 227 (6077) Control command (RETR) Request to establish the data connection Acknowledgment to the request Reply (150) data Reply (226) Receive user command Acquire port

44 (1) Overview Components of system Basic functions of system Composition, Transfer, Reporting, Displaying, Disposition Terms: UA, Mail Server, MTA address: Message Format Header, blank line, body SMTP Basic model Basic commands and replies HELO, MAIL FROM, RCPT TO, DATA, QUIT SMTP Transfer Mechanism 44

45 Components Of System (1) User Agent SMTP Mail SMTP Mail POP3 Server Server IMAP User Agent SMTP SMTP POP3,IMAP Sender Internet Spooling Mail Server of Sender Side User mailbox Mail Server of Receiver Side Receiver 45

46 SMTP Basic Model File System File System User MTA SMTP commands/replies and mail MTA SMTP client SMTP server 46

47 SMTP Commands and Status codes example Connection establish sends a mail to jones@beta.gov, green@beta.gov SMTP client TCP connection establish 220 beta.gov Simple Mail Transfer Service ready HELO alpha.edu 250 beta.gov SMTP server Mail transfer Connection release MAIL FROM: <smith@alpha.edu> 250 OK RCPT TO: <jones@beta.gov> 250 OK RCPT TO: <green@beta.gov> DATA 550 No such user here 354 End with <CR><LF>.<CR><LF> Message 221 Bye <CR><LF>.<CR><LF> 250 OK QUIT 47

48 (2) POP Basic model Basic commands and replies IMAP USER, PASS, LIST, RETR, DELE, QUIT, Features of IMAP Comparison of POP and IMAP Message formats RFC 5322: main headers MIME: New headers and main content types What are the limitations of SMTP? How is MIME used to offset the limitations of SMTP? Web-based mail 48

49 POP Basic Model Used to transfer mail from a mail server to a UA Internet File System SMTP Server: is always on and listening via TCP port 25 holds outgoing mail for user SMTP/POP Mail Server POP Mail Access Server: runs the POP3 service by listening on TCP port 110 POP SMTP User Agent Clients: connect to server to manipulate mail using message access protocols (POP or IMAP) 49

50 IMAP: Internet Message Access Protocol Features Folders and messages can be stored either on the server or on the local computer Since folders can remain on server, it is possible to access your same mail store even using a dumb terminal character based client like Pine. Much better for mobile users than POP (since mail remains on the server) Can selectively copy messages from the server to the local client based on many criteria 50

51 POP vs. IMAP(2) 51

52 WWW(1) Terminologies: WWW, the Web, W3, URL, HTML, HTTP WWW components URL Client/browser Web server The web access model Structure Used for different services HTML Static vs. dynamic CGI 52

53 Structure Of URLs A URL consists of three parts: The protocol for example http or ftp The DNS name of the host The directory and file name Protocol Hostname Directory and file name Protocol: http by default Port: 80 by default Index.html, index.htm, default.htm, default.asp etc. are assumed if no file-name given 53

54 HTTP Features Application layer protocol for client/server communication Request/response based Stateless, Transaction Main Methods Get, Put, Post, Performance enhancement of HTTP 1.1 Persistent connections Pipelining Enhanced caching options Support for compression 54

55 HTTP HTTP Transaction Establish connection Client request TCP connection set up uses a port number as application reference usually port 80 HTTP message sent with a request line request-line = method URL HTTP version Server response Connection terminated server sends HTTP message and optionally requested data resp-message = HTTP version status code reason-phrase [optional stuff] usually the server sometimes the client stops it anything else, whoever notices terminates 55

56 Example of Non-persistent connections Client TCP Connection established Server Retrieving index.htm TCP Connection released TCP Connection established Retrieving image.gif TCP Connection released 56

57 Example of Persistent Connections Client TCP Connection established Server Retrieving index.htm Retrieving image.gif TCP Connection released 57

58 Example of Pipelining Non-pipelining Pipelining 58

59 User-server state: cookies Many major Web sites use cookies Four components: 1) cookie header line of HTTP response message 2) cookie header line in HTTP request message 3) cookie file kept on user s host, managed by user s browser 4) back-end database at Web site Example: Susan always access Internet always from PC She visits specific e- commerce site for first time when initial HTTP requests arrives at site, site creates: unique ID entry in backend database for ID 59

60 Web Caches (Proxy Server) Motivation: satisfy client request without involving origin server User sets browser: Web accesses via a proxy server client Proxy server origin server Browser sends all HTTP requests to proxy server If requested file in cache: proxy server returns file else proxy requests file from origin server, then forwards to client client origin server 60

61 Conditional get cache Server does not send required files if cache has upto-date cached version cache: specify date of cached copy in HTTP request If-modified-since: <date> server: response contains no object if cached copy is upto-date: HTTP/ Not Modified HTTP request msg If-modified-since: <date> HTTP response HTTP/ Not Modified HTTP request msg If-modified-since: <date> HTTP response HTTP/ OK <data> server file not modified file modified 61

62 SNMP(1) Terminologies: SNMP, MIB, SMI, RMON Definition, goals and functional areas of network management Functional areas: FCAPS Different network management architectures and their pros and cons Centralized Hierarchical Distributed SNMP history, features SNMPv1,v2,v3, RMON1,RMON2 SNMP realizes the F-C-P functions of network management SNMP model and components 62

63 SNMP Model management station managing entity network management protocol agent data agent data managed node agent data managed node agent data managed node The SNMP model of a managed network consists of four components: Managed Nodes (Agent) Management Stations (NMS) Management Information (MIB) A Management Protocol (SNMP) managed node 63

64 SNMP(2) SNMP framework SMI and ASN.1 SMI defines the rules for describing management information using ASN.1 for an unambiguous description without inconsistencies MIB hierarchy naming SNMP protocol: traps/polling, SNMP commands 64

65 MIB ISO Object Identifier Tree Check out: This subtree is the Internet SMI

66 SNMP Traps / Polling Two ways to deliver MIB information, commands NMS NMS request response trap msg agent data agent data Managed device Polling mode Managed device trap mode 66

67 SNMP Commands Command Description Version GetRequest NMS-to-Agent: get data (instance) SNMPv1 GetNextRequest NMS-to-Agent: get data (next in list) SNMPv1 GetBulkRequest NMS-to-Agent: get data (block) SNMPv2 InformRequest NMS-to-NMS: MIB information exchange SNMPv2 SetRequest NMS-to-Agent: set MIB value SNMPv1 GetResponse Agent-to-NMS: value, response to request SNMPv1 Trap Agent-to-NMS: report exceptional event to NMS SNMPv1 67

68 GetRequest GetNextRequest SetRequest GetResponse Trap GetRequest GetNextRequest SetRequest GetResponse Trap SNMPv1 Commands Management Application SNMP Object Manipulation Managed resources SNMP Managed Objects Network Management Station SNMP manager UDP SNMP Messages (PDUs) SNMP Agent UDP Managed Nodes IP Link layer IP Link layer SNMP Device 68

69 Example of SNMP Commands (GetNextRequest) Manager Process GetNextRequest (SysDescr.0) GetResponse (sysobjectid.0) GetNextRequest (sysobjectid.0) GetResponse (syscontact..0) GetNextRequest (syscontact..0) Agent Process GetResponse (SysName.0) GetNextRequest (sysoruptime.9) GetResponse (sysoruptime.10) GetNextRequest (sysoruptime.10) GetResponse (nosuchname) 69

70 RMON RMON= Remote MONitoring Extensions to SNMP provide comprehensive network monitoring capabilities RMON uses remote network monitoring devices known as probes. The RMON specification defines a set of statistics and functions that can be exchanged between RMON-compliant console managers and probes RMON is standardized to only operate on Ethernet segments (LAN). RMON specifically defines the information that any network monitoring system will be able to provide as part of the MIB. RMON1 provides monitoring capability at data link layer in OSI model. RMON2 provides monitoring capability above data link layer in OSI model. 70

71 Real-time Services(1) Real-time services Isochronous services and QoS requirements isochronous pertains to processes that require timing coordination to be successful, such as voice and digital video transmission Jitter compensation via playback buffer RTP/RTCP Functions of RTP/RTCP, relationship between RTP and RTCP Definitions: end system, translator, mixer, monitor RTP features, SSRC vs, CSRC RTCP features, packet types, relationship of RTCP port number with RTP port number 71

72 RTP and RTCP For data RTP Important fields in header sequence number timestamp synchronization ID RTCP For QoS monitoring and control Provides feedback on the quality of the data distribution 72

73 Mixer and Translator End System (SSRC 18) 18, DVI4 18, DVI4 CSRC Mixer (SSRC=12) 12, GSM, (18,39) End System (SSRC 39) 39, L16 Translator 39, GSM SSRC:Synchronization SouRCe Identifier CSRC:Contributing SouRCe Identifier 73

74 Real-time Services(2) Multimedia signaling protocols H.323 SIP Function: Describes terminals and other entities that provide multimedia communications services over Packet Based Networks (PBN) which may not provide a guaranteed Quality of Service. H.323 protocol stack components Function: An application layer signaling protocol that defines initiation, modification and termination of interactive, multimedia communication sessions between users Architecture and components SIP messages, SIP header, SIP URL SIP Messages: INVITE, ACK, BYE, SIP Header: MIME SIP URL: user@host 74

75 H.323 Components Gatekeeper Multipoint Control Unit Terminal Packet Based Networks Gateway Circuit Switched Networks 75

76 SIP Architecture SIP Components Location Server Redirect Server Registrar Server PSTN User Agent Proxy Server Proxy Server Gateway 76