Suguru YAMAGUCHI. Nara Institute of Science and Technology Graduate School of Information Science. Information Network1 1

Transcription

1 Suguru YAMAGUCHI Nara Institute of Science and Technology Graduate School of Information Science Information Network1 1

2 Review Application model Presentation layer What is P2P? How to build large-scale services? Information Network1 2

3 OSI TCP/IP Process Unit Address Application Presentation Application Message / Stream Session Transport Transport Transport Packet Port Network Internet IP Datagram IP address Data Link Network Interface Frame Datalink Address Physical Hardware Information Network1 3

4 Layers above the Session layer are called upper layer protocols defined for each application firmly constrained to application requests Information Network1 4

5 Defines a semantic unit of communication Transaction Session Defines process for communication unit Transaction Logging & Roll-back operation Session Termination Communication conceptual definition and fundamental processing from the application viewpoint Information Network1 5

6 Data expression form This layer forms a basis to correctly express data in different platforms example expression of 1 How many bytes are used?» 1, 2, 4, less than 1 byte (6 bits),. How about byte order?» Little Endian / Big Endian How about pre-transmission bit order?» MSB first, LSB first Information Network1 6

7 Framework protocol for concrete applications note that an application protocol is not an application SMTP (simple mail transfer protocol) is defined for exchange However, there exist many applications MTA: sendmail, qmail, postfix, etc. MUA: Eudora, Mozilla Thunderbird, MS/Outlook, etc. Information Network1 7

8 Information Network1 8

9 Communication Model Client / Server model Broadcast based parallelization Peer-to-Peer (P2P) model Process Model Side effect / no side effect Process semantics Atomic / checkpoint Functions End user services Network management Information Network1 9

10 client Request/reply server client Information Network1 10

11 Easy understanding of functional separation User interface: client Each process: server function aggregation, centralized resource management manageable structure Easy service discovery model Few servers responding to many clients Service discovered by Well-known address server + wellknown port (service type Dynamic binding was difficult naming model and management model were not decided scalability maintenance is difficult currently, dynamic binding is widely used (Sun RPC bind etc.) Information Network1 11

12 The server bears all of the processing bottlenecks needs a server scale expansion to build large-scale services guarantees good scalability many various challenges for large-scale services Does not match the bidirectional information exchange model the client makes use of information located at the server active use of client-side resources and services such as client-side injections of large quantities of information is unreasonable Information Network1 12

13 Broadcast type periodical information exchange through broadcast sharing information between all processes Information Network1 13

14 Commands are sent to up & running hosts using UDP broadcast can discover active hosts process on top of host affiliated to same port performs same processing broadcast for parallel processing using hosts that exist in same segment e.g rwhod Resolve limitations of being in the same segment by using TCP server federation originated in this technology space Information Network1 14

15 server client Request/reply server server Information Network1 15

16 Direct information exchange of user-owned resources all systems are both server and client Extremely popular Gnutella, Winny,.. Issues : information discovery, security assurance of the whole system Information Network1 16

17 Widespread perception by P2P service expansion Network structure built on top of another real network at the application layer virtualization of network structure IP layer Information Network1 17

18 Communication fails Packet loss Packet duplication Complex processing due to retransmission Side effects of processing How to keep idempotence of server response? Correct understanding about identification Purpose of execution Exactly once semantics At least once semantics At most once semantics Information Network1 18

19 idempotent client always gets the same results, when the same requests are processed by the server we can build a simple retransmission system by creating such processing system pay attention to processing that alter states Information Network1 19

20 Parallelization of idempotent processing is easy Enables large-scale processing by decomposing the server inner structure look like one server The Internet Idempotent processing load balancer cluster valid for load balance and fault management Side-effect processing process aggregation target : DB etc Information Network1 20

21 Information Network1 21

22 Non structured Binary ASCII Structured TLV: Type, Length, Value ASN.1 XML Information Network1 22

23 ease of treatment / power of expression XML ASN.1 ASCII TLV binary bandwidth efficiency Information Network1 23

24 Binary IP header (RFC791), TCP header (RFC793) defines difference between host bit order and network bit order adjustment of integer value manipulation ASCII FTP (RFC959) SMTP (RFC2821) POP3 (RFC1939) ensure data transmission even for channel that is not 8-bit clean Information Network1 24

25 : 220 ftp.isi.edu NcFTPd Server (free educational license) ready : USER anonymous : 331 Guest login ok, send your complete address as password : PASS : 230 Logged in anonymously : SYST : 215 UNIX Type: L : PWD : 257 "/" is cwd : TYPE I : 200 Type okay : CWD /in-notes : 250 "/in-notes" is new cwd. Information Network1 25

26 1758: 220 ns.ixj-mc.com ESMTP Sendmail 8.9.3p2+3.1W/3.7W/ns; Fri, 30 May :15: : EHLO mf.aist-nara.ac.jp 1758: 250 ns.ixj-mc.com Hello 168.pool3.ftthtokyo.att.ne.jp [ ], pleased to meet you 0025: MAIL SIZE= : 250 Sender ok 0025: RCPT 1758: 250 Recipient ok 0025: DATA 1758: 354 Enter mail, end with "." on a line by itself 0025: Received: from mf.aist-nara.ac.jp (localhost [ ]) (JST) Information Network1 26

27 Type, Length, Value OSPF (RFC2328) RADIUS (RFC2138) Structured information expression Type-dependent processing Enables to manipulate flexible-length data by inputting Length information Value defines each expression format Information Network1 27

28 XDR (extended Data Representation) for Sun RPC/ NFS (1980 s) ASN.1 (1980 s) XML (1990 s) Requirements wire efficiency ease of treatment description static typing, name space data self-description enhancement of tools and libraries Information Network1 28

29 (tag, length, value) Tag: ASN.1 type Length: size of the ASN.1 value Value: ASN.1 value ASN.1 type INTEGER OCTET STRING OBJECT IDENTIFIER SEQUENCE (array) Information Network1 29

30 Iso(1).org(3).dod(6).internet(1) Mgmt (2) Experimental (3) Private (4) defines data structure used in SNMP MIB-II (RFC1214), Structure of Management Information version 2 (SMIv2) (RFC2578) Information Network1 30

31 <?xml version = "1.0"?> <tag attribute="value"> <another-tag another-attribute="value" /> </tag attribute="value"> Information Network1 31

32 <?xml version='1.0' encoding='utf-8'?> <soap:envelope xmlns:soap=' xmlns:xsi=' xmlns:xsd=' xmlns:soapenc=' soap:encodingstyle=' <soap:body> <n:getquoteresponse xmlns:n='urn:xmethods-delayed-quotes'> <Result xsi:type='xsd:float'>7.92</result> </n:getquoteresponse> </soap:body> </soap:envelope> Information Network1 32

33 <?xml version="1.0" encoding="utf-8"?> <SOAP-ENV:Envelope xmlns:xsi=" xmlns:soap-enc=" xmlns:xsd=" SOAP-ENV:encodingStyle=" xmlns:soap-env=" <SOAP-ENV:Body> <namesp1:getquote xmlns:namesp1="urn:xmethods-delayed-quotes"> <symbol xsi:type="xsd:string">akam</symbol> </namesp1:getquote> </SOAP-ENV:Body> </SOAP-ENV:Envelope> Information Network1 33

34 Data self-description: RDF Enhancement of tools : XPath, XSLT, Enhancement of APIs: DOM, SAX, etc. Signature, encryption XML Digital Signature XML Encryption Information Network1 34

35 Systems using XML are widely used expression in XML XML processing system XML is common knowledge for network-related service creation XML is a required item XML has been widely used for data expression in other software soon expands to a standard method Programming Language XML Information Network1 35

36 Information Network1 36

37 Warez exchange and sharing warez file exchange and sharing by ftp, www and netnews does not come under the spotlight due to its small scale Appearance of file exchange application In the early days appearance of P2P application using Gnutella and Napster Middle and late period towards a refined shape as WinMX () and KaZaa () hybrid P2P Shift to file sharing appearance of Winny (2002) towards pure P2P more effective distribution by broadband Information Network1 37

38 client / server Hybrid P2P Pure P2P center : Server center : Server peer peer peer centralized rendezvous and communication with the server rendezvous at server, communication between peers both rendezvous and communication between peers FTP, WWW WinMX, KaZaa Gnutella, Freenet, Winny Information Network1 38

39 Napster Gnutella Winny WinMX LimeWire Share BitTorrent Cabos Information Network1 39

40 Features sharing mp3 file lists between peers and enabling exchange of these files Hybrid P2P chat application was implemented to Napster as a way file exchange purpose communication tool Biography Jun, 1999 developed by Shawn Fanning (Northeastern University) Jul, 2000 sued in court by RIAA, Napster network was stopped Oct, 2003 restored as a music distribution service Information Network1 40

41 Features enables publication of not only mp3 files but also various file formats no management server and no management organization implementation of pure P2P became the reference model of P2P file sharing software Biography Feb, 2000 released by developers of Nullsoft Mar, 2000 released, but stopped release by AOL for 24 hours various clones exists Information Network1 41

42 What is Winny file sharing application successor of WinMX was developed by a Japanese individual file sharing network software using pure P2P Winny became extremely popular due to its ease of use, and a file sharing network was built Biography 6, May, 2002 β version is released 5, May, β version is released 28, Nov, 2003 arrest of a Winny user, developer s house-search, development is interrupted 31, May, 2004 developer is indicted by the Kyoto district public prosecutors office, on grounds of a Copyright Act infringement 1, Jun, 2004 bailed out 13, Dec, 2006 court-ordered at Kyoto district court, found guilty (fined), appealed the same day Information Network1 42

43 2003 estimated 200,000 (network company survey) 2 millions (ACCS survey) users Nov, 2004 domestic traffic was reduced by 1/6 July, 2006 estimated 1.75 millions user Information Network1 43

44 Issues on the quality of the exchanged information distribution of copyrighted works distribution of counterfeited files and faked files spread of viruses Arising from file management due to the use of file as the information unit anyone can freely share a file can not delete a shared file impossible to identify the sharing source of a file has no way of knowing the reliability of a targeted file needs to exchange more structured information Information Network1 44

45 Potential customers has proved existence of a consumer layer that can be contents purchasers given appropriate price and availability Construction of an autonomous distribution network by users showed new solution for maintenance of distribution network possible information distribution infrastructure by introducing appropriate technologies Information Network1 45

46 Expectations for P2P Distribution Network cost reduction due to flow disintermediation direct distribution from creator to user distribution maintenance of minor products maintenance of distribution market Requirements for P2P distribution network File preservation User preservation Accounting Traceability of file Information Network1 46

47 Aggressive use of P2P network Oct, 2005 Nasper Japan is established by Tower Record and Nasper Kazza create business model by the distribution of promotion files, the delivery of pay files and advertisement Skype voice communication service by Gnutella technologies is widely used all over the world Information Network1 47

48 Segregation between physical network structure and application layer structure appearance of usable bandwidth stress state Is it possible to build overlay-friendly underlying networks? We can get a hint by elaborating overlay networks Research is also progressing on what kind of hinds can be obtained FY2010 I 12 48

49 E.g., in the case of an information storage service built as an overlay network. Which node hold the desired information? There exist many issues Potential search halt Does the search has been completed within the configured time? Is there any efficient information search method? Existence of hindering nodes What to do if some nodes do not relay the search? It is assumed that each node is not under the provider control What to do when search results include down nodes? FY2010 I 12 49

50 Information Network1 50

51 a few servers for many clients the whole is visible as a single server The Internet Idempotent processing load balancer cluster valid for load balance and fault management Side-effect processing process aggregation target : DB etc Information Network1 51

52 Required mechanism for load balancing Load balancer is widely used Operating principle LB manages termination point of TCP connection Server1: Client: The Internet Virtual server: Server2: Forwarding Table Src: Dst: [1-3] Server3: Information Network1 52

53 Which servers ended the connection? Various methods were designed and implemented round robin, minimum connection, performance observation fault management is also important how to detects when a server is down? How to hide the difference of server addresses When server-side uses a private IP address, we must rewrite address in communication No simple NAT sometimes address information is embedded into Cookie or HTTP payload some application protocols has side effects Information Network1 53

54 Server does not need to exist in the same network show multiple servers as a single one by using tunneling sometimes this technology is called server federation CDN (Contents Delivery Network) active load balancing has been generalized in ISPs Akamai.com is the pioneer of business service about CDN. Information Network1 54

55 server IX clients Information Network1 55

56 new contents are provided by server at data center(freshness control) Data Center clients individual ISPs provides contents for customer by CD server (well-managed access) Information Network1 56

57 load balance by routing information Root DNS server is a typical example Information Network1 57

58 HOST 2001::1 HOST HOST HOST 2001::1 2001::1 Information Network1 58

59 F root DNS Originally at ISC in Silicon Valley, Calif. Currently, Hong Kong & Amsterdam??? client request reaches the neighbor server that is on the shortest path depending on network routing Hong Kong, CN each server, while using the same address, distributes route information from each location. Amsterdam, NL Silicon Valley, Calif. Information Network1 59

60 FY2010 I 12 60

61 Real-Time Media Transmission Technology for voice and video transmission Real-Time Packet Transmission over Network Layer and Transport Layer Coding Techniques which are adapted to media characteristics at the upper layer Providing and sharing SDK (Software Development Kit) for building real applications FY2010 I 12 61

62 Internet: to support the real economy The Internet is a social infrastructure for trading money The security technologies become essential for the protection of assets Encryption, Authentication Technology, ID Management Technology Secure Distributed Information Management Service Backup for BCM (Business Continuity Management) Standardization of application protocols implementing the above services has started. FY2010 I 12 62