Large-scale Archival Storage - a brief overview for the HEP use case -

Large-scale Archival Storage - a brief overview for the HEP use case - GDB, 13/9/2017 Germán Cancio CERN IT/ST 1

Agenda Status of tape market and technology Alternatives to tape Disk Optical Holographic 2

Tape Market dominated by LTO consortium (~95%) IBM, HP, Quantum (drives) + Fujifilm, Sony (media) Oracle resells LTO drives from IBM Enterprise tape (IBM+Oracle) ~4% IBM: Latest: TS1155 @ 15TB, 350MB/s introduced May 2017 Oracle: Latest T10KD @ 8TB, 250MB/s introduced Sept 2013 Tape drive head technology: From GMR to TMR GMR has reached its density limits (HDD s: stopped in 2004) TMR requires substantial R&D and manufacturing retooling TS1155 uses TMR, LTO-8 will use it Large-scale libraries (>=10K slots) Oracle, IBM, Spectra Logic, now also Quantum 3

LTO and IBM enterprise tape roadmap (source: IBM) Expected EOY 2017 Released Q2 2017 8/6/2015 DPHEP Collaboration Workshop 4

Oracle enterprise tape roadmap 5

Tape drive head manufacturing (Source: Spectra) 6

Tape drive head manufacturing (Source: Spectra) 7

08/2017: Sony/IBM demo (using CoPtCr) 201Gb/in 2 ~330TB tape IBM TS1155 9.6Gb/in 2 04/2015: Fuji/IBM demo (using BaFe) 123Gb/in 2 ~220TB tape

Tape Market evolution LTO media shipments decreasing since ~2007 Consolidation, competition of disk and cloud solutions LTO media units shipped, 200 2016 (source: LTO consortium) ~ 20M units per year (~40 EB) Media price ~10-15CHF/TB(*), but decay has slowed down (-20%/yr over last 4 years) Two remaining media manufacturers (TDK exitus 2014) (*)Media is ~50% of tape TCO (add drive and library HW, maintenance) SPoF as IBM only remaining (major) manufacturer + R&D of tape drive technology Will this market sustain to drive (enterprise and LTO) tape research (new heads, new media) and production? 9

Disk (Spinning) disk market: WD (41%), Seagate (37%), Toshiba (22%) ~600EB/year, decreasing since 2010 Nearline (high capacity, high quality) drives used in HEP: ~10% of market sales Increased competition from cloud & SSD for notebooks and enterprise disks Source: Statista / B.P.S. Source: Wikipedia 10

Disk Technology Shingled recording disks (SMR) ~2013 Helium filled disks (more platters) ~2013 HAMR disks not before 2018(?) Complex technology, laser+new media, reliability+cost are issues Capacity evolution 14-16TB in 3-12 months 20TB in 2020 100TB by ~2025 with HAMR/HDMR Pricing for nearline disks decreasing ~14%/year, currently ~35-40CHF/TB Very shaky price evolution SSD s vs HDD s? Shipped capacity in 2016: ~45EB (roughly 7.5% of shipped HDD capacity expected to grow to ~20% around 2021) Large investments required for SSD manufacturing (200-300B$) SSD/TB price for capacity in foreseeable future still expected O(10)x of disk/tb price 11

Disk Servers for Archival? Current CERN EOS disk servers: one CPU node + 2x24 enterprise-class capacity disks. JBOD with 2 replica. Disk cost is 75% TCO is ~3x of tape CHF/TB Looking into optimising CHF/TB to close gap with tape. Ongoing investigations: Using desktop disks à la BackBlaze (30%-40% cheaper) No warranty / certification for our use case Measure reliability of different models/vendors (integrity, failure rates) using SMART and EOS monitoring Compensate lower reliability with higher redundancy Review operating procedures: Let disks die rather than replacing Two servers in production for ALICE (successfully so far) Monster servers for optimising disk-to-infrastructure cost ratio Testing 192 disks (2 trays with 8x24 HDD s each) on one server Up to 1.1PB raw with 6TB disks Evaluate different file system and redundancy layouts (ZFS pools, RAID, EOS erasure encoding) -> f(capacity, reliability, performance) 12

Optical Archival Disk evolution of Blu-Ray Collaboration between Sony & Panasonic Max capacity 300GB disks (WORM only) 140MB/s write, 280MB/s read Reliability (Blu-Ray UBER 10-12 ) -> erasure coding overhead Roadmap to 1TB but no timeline nor new products 1TB by now 2020 (was 2010, then 2012..) Consumer market for optical disks vanishing Behind magnetic storage in capacity, performance, reliability Media volumes, pricing? Cheaper than HDD Libraries announced by Sony (Everspan) and Panasonic (Freeze-Ray) Robots mounting media trays for 4x16 disks (Everspan) / 12 disks (Freeze-Ray) Up to 14 expansion media racks @ 13PB raw (Everspan) -> ~180PB Up to 64 drives / library (Everspan) Customers? Everspan evaluation started by LANL 14

Holographic reference beam CCD readout Record information across media volume, not just surface High densities using different recording angles, wavelengths, position on single media location Potentially, O(GB)/mm 3, fast R/W rates 2015 Demo: 2Tb/in 2 -> ~770GB/in 3 signal modulation media Prototypes, even ECMA standards 300GB/disk, 20MB/s Small companies: InPhase (bankrupt in 2011), Akonia Holographics Research @ IBM Almaden Labs (until ~2000); GE No products on the market, nor any signs of upcoming ones 15

Summary Tape is still the most cost-effective archival solution for HEP, but concerns about the long-term sustainability of a contracting tape market dominated by a single technology provider Disk market also contracting but from a wider base in terms of volume and vendors Possible opportunities by exploring massive cheap disk setups for archival in order to close the 3x cost gap wrt tape Optical archival storage has recently seen a kind of revival, but no noticeable market impact yet Will holographic ever surface? 16

References Storage Technology and Markets (B. Panzer-Steindel, CERN IT CTO) IBM enterprise and LTO product roadmap Spectra Logic presentation to the DMF UG, 2017 INSIC consortium tape technology roadmap 2015-2025 LTO consortium 2016 tape capacity shipments Clipper Group TCO study ASTC disk technology roadmap, 2016 Monster node testing @ CERN/IT: Roberto.Valverde.Cameselle@cern.ch BackBlaze blog: Storage Pod Sony Everspan specs Akonia Holographics 2Tbit/in 2 press release 17

Source: IBM 18