Massimo.Carboni@garr.it Garr-X Progress: a new step forward on GARR optical network Gloria Vuagnin Paolo Bolletta 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th
OUTLINE GARR the dream (i.e. the overall design) 2012: GARR-X - the basic national optical network footprint Transport technology Lesson learned 2014: GARR-X Progress - extension in South Italy Main objectives Footprints Transport technology Timing Integration and future what next? 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #2
Design optical fibre footprint Blue color: implemented at first phase (2012) Gray color: not implemented at first phase Italy Shape does not help to have uniform coverage Network topology mesh Good/acceptable in Nord- West and Central Poor in Nord-East, South, Insular 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #3
GARR-X Network (phase1-2012) 1 2 ) ) 1 Backbone Fibres Some integration with leased lines Access 10G (co-located) 10M->1G Leverage on MANs Leased fibres Leased lines 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #4
GARR-X Transmission Equipment OLA OSN6800 Total number of ports: Total number of nodes: 81 32 x ROADM; 23 x OTN 49 x Amplifier 1R (OLA); Average span length: 71 Km Longest span ~ 130 km (Bo Fi) BER target value: 10-16 post-efec Pre-FEC BER @ 10G < 10-15 Pre-FEC BER @ 40G < 10-9 No-Coherent 10G/40G DCM Upgrade 100G Coherent Network boards: 155 ND2 (2x10G) + 10 NS3 (1x40G) Client boards: 41 TOA (8x multirate; 328 ports but 242 sfp) + 63 TQX (4x 10G; 253 ports but 232 xfp) ROADM OSN8800 T32 Subrack 3 6-degree node 3 5-degree node 5 3-degree node 15 2-degree node 4 1-degree node 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #5
Huawei solution: overall good, however Learned Lesson Robust and flexible technology, however: To grow in speed is a cost Substantial overhead to 100Gb/s evolution Lack to understand our point of view Operations Time Consuming Non-optimal optical power adjustment Restoration implies constant tuning activity WSON: works perfectly when check on rerouting path is done periodically Raman pumps: to handle with care Fibres cuts, Reflection Sensitivity,. Too Long Delivery Time Low level skills in Local support team Professional skills for complex task ONLY on Chinese experts Fibre footprint. Limited effect coming from fibre mixture G.652/G.655 Guard band (10/40/100) 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #6
GARR-X & GARR-X Progress (2015) GARR-X Progress is the extension of GARR-X in the South of Italy 1 2 ) ) GRANT fund from Italian R&E Ministry to remove the digital divide in the South 1 Network infrastructure ITC (end user s services) Training Constrain: expenditure before March 2015 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #7
GARR-X Progress Project Requirements A shared High Speed Network for the community of: Schools Universities Research Applications to delivery widespread IT services to the R&E Community Users Community Service e-infrastructure Packet Layer Circuit Layer Physical Layer 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #8
GARR-X Progress Pillars Fibre for everyone Backbone Access Schools High Speed Transport Network DWDM Coherent vs Terabit Network Solution Superchannel P-t-P with services at: 10G/40G/100G OTN Restoration A2B: Traffic Matrix @ 10G/40G/100G A2A: Traffic Matrix @ 10G/100G Access on fibre Schools 100M on 1G link Access 1G over 10G link High End 10G over low expensive DWDM solution ICT (end user s services) Distributed Data Center 5 PoPs High level Service: IaaS / PaaS / SaaS End User Service: GARRBox (on 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #9
Planned Initial Capacity Banda Ag Ba1 Br Cs Ct1 Cz Fg Le Maz Me Na1 Na2 Na6 Pa1 Rc Rg Sr Ta Tp A->B Ag 0 0 0 10 0 0 0 0 0 0 0 10 0 0 0 0 0 20 Ba1 20 0 280 10 10 20 0 20 360 40 0 180 0 0 0 10 0 950 Br 0 0 0 0 0 0 0 20 0 0 0 0 0 0 0 0 40 Cs 40 0 0 0 0 0 40 0 0 0 0 0 0 0 0 80 Ct1 0 0 0 10 0 320 0 0 180 0 10 30 0 10 890 Cz 0 0 0 0 10 0 0 0 0 0 0 0 0 20 Fg 0 0 0 10 0 0 0 0 0 0 0 0 20 Le 0 0 20 0 0 0 0 0 0 0 0 40 Maz 0 0 0 0 10 0 0 0 0 0 20 Me 0 0 0 20 0 0 0 0 0 40 Na1 40 20 0 10 0 0 10 0 860 Na2 20 0 0 0 0 0 0 100 Na6 0 40 Pa1 10 10 10 0 10 440 Rc 0 0 0 0 20 Rg 0 0 0 20 Sr 0 0 40 Ta 0 20 Tp 20 3680 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #10
The Progress Transmission Network Na 1 Na 2 Fg 1 Fg Na 6 Ba 1 Br Ta 1 Le Cs Cs 1 Cz Rc Me Ct 1 Pa 1 Sr Tp Rg Ag Maz 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo.Carboni@garr.it (GARR) #11
Switch Slots Infinera Solution 16-OCG @500Gbps - 8Tbps max capacity - 100Gb/s client Tributary Modules Tributary Modules (xtm) support 10 x TIM slots TIM modules for client interfaces: Sub 10 G, 10G, 40G & 100G <10G service through sub 10G TIM and aggregation shelf 16 x 2.5G 5 x 10G 1 x 40G 1 x 100G Universal Slots Switch Slots Capacity model: pay as you go Line 500Gb/s p-2-p Client 10G/40G/100G Switch Modules Up to 5Tb/s switching per bay Multiple switching options: OTN muxing/switching (OXM) Hybrid packet/otn (HXM) DWDM Line Modules PIC-based 500Gb/s DWDM line modules (AOLx-500) 5 x 100Gb/s with FlexCoherent modulation (QPSK, BPSK, ebpsk) ULH reach up to 2800km+ Universal Slots Universal Slots XTC Chassis Ten (10) universal I/O Slots Mix & Match Trib / Line Modules Universal Slots 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #12
The IP/MPLS (2012) x100ge x40ge x10ge Core T Access PV TS 1 MI 3 TN MI 2 MI 1 GE TO 1 PI 1 MI 2 MI 2 MI 1 PEERING NATIONAL MI 1 PD 1 PD 2 VE-2 SS CA 1 PEERING RESEARCH BO 3 RM 1 GLOBAL INTERNET FE RM 4 RM 2 RM 2 BO 1 BO 1 UR PG RM 2 BO 1 AN AQ FRA FI 1 NA 1 CT 1 PA 1 BA 1 SA ME 1 BR LE CS MT PZ 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo.Carboni@garr.it (GARR) #13
The IP/MPLS evolution x100ge x40ge x10ge Core Access NA6 NA 1 FG SA BA 1 LE BR NA2 TA1 NA 1 BA 1 DC RC CS T 2 T 2 DC CZ ME ME 1 CT 1 T 2 DC DC PA 1 CT 1 PA 1 PA 2 Mx960 with @ 100G/40G/10G Mx480 with @ 40G/10G/1G 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo.Carboni@garr.it (GARR) #14
GARR-X Progress vs GARR-X Numbers comparison GARR-X GARR-X Progress Backbone 6500 km + 3000 km Access 1500 km + 2000 km Schools 0 km + 1000 km WDM Nodes 32 + 21 IP/MPLS nodes 38 + 8 Client Capacity 1 Tbps + 5 Tbps vcpu (E5-2697v2) 0 +8800 Storage (PB) 0 +10.3 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #15
Where we are now everything in parallel GARR-X Progress Procurement Procedure: [1301] DF 07/13 [1401] DWDM 02/14 [1402] IP 03/14 [1403] ICT 04/14 GARR-X Progress is now on delivery phase: Fibres: delivery at 80% Routers: before end of November WDM: end of year ITC: within this year 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #17
60 GB The Whole IP-Network 60 GB 120 GB 60 GB 100G Evolution 14 Upgrade x100ge x40ge x10ge Core T Access 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo.Carboni@garr.it (GARR) #18
Integration and Future evolution from South North SWITCH Mi 2 Mi 1 CERN RENATER ACONET ARNES 1 2 ) ) ALBANIA 1 GREECE Malta 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #19
The Whole IP-Network 100G Evolution Full Upgrade x100ge x40ge x10ge Core T Access 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo.Carboni@garr.it (GARR) #20
To do What?
Fibre to the Schools GARR and Schools Give schools a tool for teaching Represent an alternative network (compared with the Operator approach) Only High Schools (colocated with the GARR POPs) GARR Network Model Acquisition of Dark Fibre (IRU 15y) Management Only IP Demarcation Point Punti di demarcazione del servizio In fase di valutazione Services Symmetric connection @ 100M (1G ready) Public IPv4 and IPv6 address space No filtering and limitation Riunione CCR - Roma, Presidenza INFN - Roma - 8/settembre/2014 Massimo Carboni (GARR) #22
Schools Aggregation Model 100 Mbps 1 Gbps Aggregation 10 Gbps Service Edge CPE Access Node Aggregation Node Service Node Aggregation 100 Mbps Fibre 100 Mbps Radio 1 Gbps Fibre 10 Gbps Fibre 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #23
ICT Logical Infrastrutture Cloud Services VM Repository Virtualisation Level KVM Translator VM Virtual Disk System Hardware Level 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #24
ICT Components & Basic Service 11 Racks with Power Consumption >15kWh/Rack Item Quantity Technology Comments Computing 8800 Blade systems with 11 Blades 384 GB Memory x 2 Processors x 12 Cores system vcpu L2/L3 and FC x 2 HyperThreading = 512 Virtual CPUs + FiberChannel and 40 GB Ethernet Switches. Disk system 10 PB RAID6 disks in FC 10 PB storage with redundant FC links (4 controllers x 2 8Gbit/s links each) and servers + FC Switches. Servizi Base su Cloud BigDataStorage-Small BigDataStorage -Medium BigDataStorage -Large GARRbox-Small GARRbox -Medium GARRbox -Large Volume 100GB 500GB 1TB 10GB 20GB 200GB Virtual Server vcpu vram vstorage (GB) (GB) vserver-small 1 2 100 vserver-medium 2 4 250 vserver-large 4 8 500 vserver-extralarge 8 16 500 vserver-extralarge-32gb 8 32 500 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #25
Cloud Controller App1 App2 2 3 1 3 CLOUD MONITORING Site A Site B Site C 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #26
From Davide Calonico, PhD @ (INRIM) Optical Division METROLOGY APPLICATION
Atomic clocks accuracy in the last 50 years First Cs beam clock (1955) 10-8 10-10 First Cs fountain (1996) 10-12 Cs fountain clocks (today) 10-14 10-16 Satellite techniques (1 day averaging) Best optical clocks (today) 10-18 10-20 Optical links (1 day averaging)
Towards an Italian Optical Fiber Network for high accuracy Time&Frequency Metrology MILANO, IFN-CNR TORINO, INRIM BOLOGNA, INAF First Step: INRIM (TORINO) to LENS, (FIRENZE), 650 km optical fiber link, Operational from late 2013 FIRENZE, UNIFI LENS, INO-CNR
INAF MEDICINA RADIOTELESCOPE CNR-LENS-UNIFI ATOMIC PHYSICS ROME-FUCINO, METROLOGY, SPACE, GALILEO INAF SARDINIA RADIOTELESCOPE MALTA PRIMARY METROLOGY MATERA, SPACE GEODESY INAF NOTO RADIOTELESCOPE
LIFT impact on research Frejus Relativistic Geodesy Florence Atomic and molecular Physics Bologna Radio-astronomy Very Long Baseline Interferometry Cagliari Matera Noto Space Geodesy Center Bepi Colombo
EMRP Project: NEAT-FT (2012-2015) 2014-2015 2014-2015 2015-2016
Conclusions Managing extra fund in a short time frame is not simple The experience of GARR-X help us to adopt some shortcuts: administrative and technical From the competition comes new opportunity at reasonable price Fibre is the solution adopted for the integration To support new applications and new users community The evolution of the network continue. The next step is fibre for everyone on the same duct Backbone, Access and other users like metrology In case of schools we need to adopt a planb, the system is not ready Two Optical layer will be used Access Optical Layer @ low speed with Lambda @ 10G, 40G Services @ 1G/10G/40G Backbone Optical Layer @ High Speed with SuperChannel @ 500G, 1T or more Services @ 40G/100G/400G (~ 2017) The Computing Model of LHC experiments could consider the WAN as sort of distributed LAN (or a multidomain LAN) Esnet show us the requirement of the HEP community Science DMZ 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) 33
BACKUP SLIDE
Network Comunication App 1 App 2 MONITORING MONITORING Network Layer Network Bridge Network Layer Infrastruttura Hardware (CPU+Storage) Network Virtualization Infrastruttura Hardware (CPU+Storage) Storage Layer Storage Syncronization Storage Layer 8th Customer Empowered Fibre Networks Workshop, Prague, September 15th-16th Massimo Carboni (GARR) #35