The Status and Perspectives of the JINR 10 Gbps Network Infrastructure

Transcription

1 The Status and Perspectives the JINR 10 Gbps Network Infrastructure K.N. Angelov, A.G. Dolbilov, A.E. Gushin, V.V. Ivanov, V.V. Korenkov, L.A. Popov Laboratory Information Technologies, JINR, Dubna Introduction The JINR Gigabit network infrastructure is an aggregation the multifunctional network equipment and specialized stware, which form the basis for the advanced information and computing JINR infrastructure. The main goals the JINR network infrastructure are as follows: creation the common informational space the available computing and storage resources in JINR; creation the common information space for JINR personal, providing the possibilities for the data exchange between JINR divisions; organization and provision the network access to the JINR central resources and and resources located in the JINR divisions; support the availability the JINR Grid segment, as a part Russian Data Intensive Grid (RDIG) infrastructure; provision the access to the INTERNET resources; support the common network services: (SMTP, IMAP, POP3, WebMail), file exchange (ftp, scp, sftp, http, https), security (ssh, https, RADIUS and TACACS authentication), DNS, users accounting, maintenance the networks elements - IPDB, etc. The JINR network infrastructure consists the following components: the external JINR-Moscow optical telecommunication data channel; the JINR backbone network; local networks the JINR divisions. By the JINR Administration mandate the Laboratory the Information Technology is responsible for maintaining the external channel and the JINR optical backbone. Local area networks in the JINR divisions are supported by specialists from corresponding divisions. The JINR telecommunication channel The external JINR Moscow optical telecommunication channel uses 10 Gbps Ethernet based on the DWDM (Dense Wavelength Division Multiplexing) technology. The most attractive feature the DWDM technology comparing with the traditional optic technologies which transfer one data signal over a single fiber is parallel transfer the multiple digital signals over a single fiber. To achieve this remarkable property, a digital signal stream is parsed into multiple chunks which are to be sent over a single fiber by multiple virtual optics channels, called lambdas (λ). Modern state the DWDM allows up to 160 λ-s. Each λ has its own frequency value, and neighbor λ-s do not overlap, giving the possibilities simultaneous parallel data transfer. The equipment OME-6500 and CPL the Nortel Systems company (Canada) based on the DWDM technology were installed in three places in 2009 to form the JINR Moscow external cnannel: in the central telecommunication node in LIT (r.215, bld.134), in the settlement Radischevo (130 km away from Dubna), in the Moscow Internet Exchange MIX (Moscow, Butlerova Street). Figure 1: The scheme the telecommunication channel between the JINR LAN and MIX NAP Figure 1 presents a scheme the telecommunication channel between the JINR LAN and MIX NAP (Moscow Internet Exchange Network Access Point). The optical cable passes three intermediary points: Konakovo, Radischevo and Starbeevo; the active network equipment is installed only in one these points in Radischevo. This particular type the equipment works with λ = 10 Gbps. We are using two λ-s, which means that at the current moment the JINR telecommunication optical channel has a capacity 20 Gbps in total. Currently, one λ is in use for the e-arena project, and the second one is planned to be used for providing the virtual data channels in the JINR ISP project. It must be noted that with this type the optical hardware we are able to increase the number λ-s up to 88 with the total channel capacity 880 Gbps. The JINR Local Network has direct connections 11

2 to the following scientific, educational and public networks: GEANT (Gigabit European Advanced Network Technology) main European multigigabit computer network for research and education purposes 10 Gbps, RBNet Russian Backbone scientific network 10 Gbps, Moscow and Saint-Petersburg scientific and educational network 10 Gbps, Internet 10 Mbps. The intercity data communications The intercommunications between different networks in Dubna city are organized on the basis the specialized data exchange node DBN-IX in JINR central telecommunication node. This solution allows the Dubna Internet providers to organize their traffic transfer in direct way instead sending it to Moscow Internet Exchange and then back to Dubna to the partner. JINR Local Network has direct connection to the following networks: Netby-Net (the former LAN-Polis) 1 Gbps, Contact 1 Gbps, Telecom-MPK 1 Gbps. The VPN-based network remote access is available for the JINR employees from their homes by the Net-by-NET (the former LANPolis), Contact, and Telecom MPK networks. It should be noted that the internetwork peering for the city ISPs is organized. The state the JINR Local Network The JINR Local Area Network (LAN) is referred to as a campus-type network (a campus is an extended student town or a research center with multiple distributed buildings). The JINR LAN comprises an explicit backbone in the form three rings to reach different parts the JINR LAN and access networks. The access networks in JINR are users networks which serve the scientific and administrative divisions; they are maintained and developed by the staff the divisions. There are a number L3 switches (Cisco Catalyst 3560-E) installed in the backbone to serve as the input/output gateways for the corresponding divisions. These switches being part the JINR optical transport backbone physically are located in the JINR divisions, but they are under control the JINR LIT network operations center. Access switch ports which face the internal divisions structures are under control local division staff. Thus, the backbone switches, the optical transport cable, the equipment the central telecommunication node, the external telecommunication optical channel are under supervision the LIT Network Operations Center (NOC). Since the mid 2010 the JINR network has been using two data speeds, or data rates. Figure 2: The current state the 10 Gbps JINR LAN backbone Three Laboratories (LIT, LNP and LHEP) use 10 Gbps data rate (Fig. 2), the rest divisions use 1 Gbps data rate. A complete 10 Gbps backbone will be ready till the end the year The monitoring the JINR network The JINR networking monitoring system uses a few protocols: SNMP, NetFlow, accounting. The SNMP (Simple Network Management Protocol) picks up the necessary network parameters, such as, for example, state the network device interfaces, the loading these interfaces, the activity/inactivity state the network elements, the bandwidth the data communication lines, etc. The NetFlow protocol picks up the traffic data on the basis hosts IP-addresses. The accounting is a more simple protocol the network control, aimed to present the events in the real-time approach and to be used for the logging the network events synchronized with real time. The results the monitoring are presented by the Web-interface, called NMIS Network Management Information System. The NMIS package is distributed on the terms the GPL (GNU Public License) usage. It was adapted for the use in the JINR networking environment. The NMIS provides the following services: permanent monitoring the JINR network state; presentation the general network state and the status main network elements (routers and switches); showing the status and the usage the critical JINR network elements. The control (monitoring) the networking events consists the following elements (subsystems): data collection system the control information (SNMP, accounting, and netflow); storage system for the collected data information; system processing and visualization the stored information. 12

3 Figure 3: The structure JINR local area network Figure 3 gives a structural view the JINR 10 Gbps Ethernet network, which includes Cisco 7606 edge routers with optical links (upper lines in red) to the JINR ISP in Moscow, two central core Layer 3 switches Cisco Catalyst These two 6509 switches drive all layer 3 backbone Cisco 3560E switches, installed in the JINR LAN backbone at every main division (9 divisions in total). Besides, the above mentioned two flow collectors (the primary one and the backup) are connected to the Catalyst 6509 switches for the traffic analysis. Figure 4 shows the screenshot the IPDB system Figure 5: An example the NMIS screenshot ploying the sophisticated JINR LAN structure through its own WEB-interface. The following are the activities with IPDB (authorization, authentication, and accounting): registration and authorization the JINR network elements and JINR Network users; visualization the JINR network traffic statistics; maintaining the VPN data base remote users; holding users data base electronic libraries; working with users data base the computational JINR cluster. Figure 4: An example the IPDB screenshot which demonstrates the amount data transferred through the JINR network in Data are presented as incoming into LAN, outgoing from the LAN, and all the data are differentiated by the division they relate to. Figure 5 presents the status the particular node (host). The JINR LAN authorization system The authorization system is one the most important systems for the network any organization, it is one the crucial elements in the organization security system. The authorization and accounting in JINR are organized on the basis IPDB. It uses the following protocols: LDAP, Kerberos, RADIUS, and TACACS. The IPDB is the JINR network nodes data base structure used for the de- Figure 6: The detailed information related to a particular node (host) and information about a user, logged in to the node Figure 6 presents the detailed information related to a particular node (host) and information about a user, logged in to the node. Statistics based on IPDB and NMIS The most interesting statistical data related to the networking can be seen below. These data are withdrawn from IPDB and NMIS packages. Table 1 shows the contents the IPDB database, and Table 2 shows the external incoming to JINR Network traffic. 13

4 Table 3 shows the data for the 2011 year distribution the traffic by the categories. Table 1: Content the IPDB database operator (administrators 43 the JINR divisions) users 3530 network elements 7050 IP addresses 8495 remote access (total/ per 1188 / 15 month) electronic libraries (total/ 964 / 20 per month) AFS users 365 IPDB transactions, thousand 160 per year number the scientific 270 networks Table 2: The incoming traffic to the JINR network year incoming traffic, Tbyte Table 3: The data for 2011 year distribution the traffic by the categories N Category/ Alias 1 Scientific and educational organizations/ SCIENCE 2 File Exchange (p2p)/p2p 3 Web resources/ WEB 4 Social networks/ SOCIAL NET 5 Stware/ SOFTWARE 6 Multimedia flows/ MM STREAM 7 Dubna networks/ DUBNA Incoming (IN)JINR Outgoing (OUT)JINR IN OUT Total: Table 4 shows the Top 10 data for 2010 for scientific and educational organizations by subcategories. Dynamics the JINR external data channels capacity is shown in Table 5. Dynamics the JINR LAN data rate is shown in Table 6. Conclusion The main tasks the development the JINR Table 4: The top 10 data for 2010 year for scientific and educational organizations by subcategories N Category/Alias Incoming (IN)JINR 1 Science Park Watergraafsmeer Amsterdam (NL)/SARA 2 Nationaal instituut voor subatomaire fysica (NL)/ NIKHEF 3 Fermi National Accelerator Laboratory (US)/FNAL 4 Istituto Nazionale di Fisica Nucleare (IT)/INFN 5 European Organization for Nuclear Research (SE)/CERN 6 Forschungszentrum Karlsruhe (DE)/ KFK-ULTRA 7 Rutherford Appleton Laboratory (GB)/RAL 8 Institut National de Physique Nucleaire (FR)/IN2P3 9 Consorci Institut de Fisica Altes Energies (ES)/PIC 10 Academia Sinica Computing Centre (TW)/ASCC Outgoing (OUT)JINR IN OUT Total: Table 5: Dynamics the JINR external data channels capacity Year Chan. cap., Mb/s x10 4 2x10 4 Table 6: Dynamics the JINR LAN data rate Year Data Rate, Mb/s network infrastructure fulfilled in are as follows: operational solution the complete 10 Gbps backbone the JINR local area network; the approach for the provision the external customers with the virtual data channels. The basis the virtual service is a fault-resistant design the network structure: hardware, stware, the means the on-the-fly diagnostics, on-line monitoring the network health. The second important part this task is the obtaining a specialized license for fering the virtual channels for corporate users in Dubna. The long-term perspectives the JINR networking development is the quantitative growth the main networking parameter data rate. As it was 14

5 already mentioned, the aggregate speed the external data channel can reach 880 Gbps. The speed the JINR data backbone will be increased, but not so quickly as the external data channel. Also we do not expect in the near future that the interfaces the workstations will become equal to 10 Gbps. More attention should be paid to the all JINR networking aspects to be able to satisfy demands the users, who utilize more and more sophisticated applications in their everyday work. The efforts the LIT NOC specialists should be applied to the following: to support the exploration and extension the multimedia applications for the JINR users, to integrate the low-level network monitoring and the Grid monitoring to have the fast and reliable problem-solving mechanism, improvement the reliability, availability, and quality the network, the external data channels, and all provided network services. References [1] Э. Таненбаум. Компьютерные сети. СПб.: Питер, 2003, 4 изд. 948 с.: ил. [2] В. Столлингс. Современные компьютерные сети. СПб.: Питер, 2003, 2 изд. 783 с.: ил. [3] Безруков Б.А., Долбилов А.Г., Гущин А.Э., Емелин И.А., Медведь С.В., Попов Л.А.: Опорная сеть ОИЯИ на технологии Gigabit Ethernet, In: Annual Report 2003, Laboratory Information Technologies. Ed by Gh. Adam, V.V Ivanov and T.A. Strizh, JINR, Dubna, 2004, c [4] Ангелов К.Н., Безруков Б.А., Гущин А.Э., Долбилов А.Г., Емелин И.А., Иванов В.В., Медведь С.В., Попов Л.А.: Гигабитная сетевая структура ОИЯИ, сетевые сервисы, сетевая безопасность, In: Scientific report , Laboratory Information Technologies. Ed by Gh. Adam, V.V. Ivanov and T.A. Strizh, JINR, Dubna, 2005, c [5] K.N. Angelov, A.G. Dolbilov, I.A. Emelin, A.E. Gushin, V.V. Ivanov, V.V. Korenkov, L.A. Popov, V.P. Sheiko, D.A. Vagin: JINR networking results, In: Scientific report , Laboratory Information Technologies. Ed by Gh. Adam, V.V. Ivanov, V.V. Korenkov, T.A. Strizh, P.V. Zrelov, JINR, Dubna, 2007, pp [6] K.N. Angelov, A.G. Dolbilov, I.A. Emelin, A.E. Gushin, V.V. Ivanov, V.V. Korenkov, L.A. Popov: JINR networking results, In: Scientific report , Laboratory Information Technologies. Ed by Gh. Adam, V.V. Ivanov, V.V. Korenkov, T.A. Strizh, P.V. Zrelov, JINR, Dubna, 2009, pp