TechTour. Winning Technology Roadmaps. Sig Knapstad. Cutting Edge

Transcription

1 TechTour Winning Technology Roadmaps Sig Knapstad Cutting Edge 1

2 2

3 Avid Nexis E5 3

4 Spectra BlackPearl TechTour 4

5 Simplified Overview T950 5

6 6

7 Winning Technology Roadmaps TechTour What does your storage landscape look like? Do not delete anything is the new mantra 30% more storage companies than last year 7

8 ZETA-WHAT?

9 If you ask Brian or Zeke Requirements: Storage Considerations-speed-cost-capacity Data Considerations-business value-recall probability-retention Need to match the storage requirements to the storage resource This has become more complex as new forms of storage have been introduced 9

10 Storage Architectures Available Today When to choose Object over SAN or NAS? TechTour 10

11 Block Storage TechTour Files are split into evenly sized blocks of data, each with its own address No additional information (metadata) The client is loaded with specialized software that is responsible for presenting the local file-system Everything is handled and controlled by the SAN software SAN file-system Ideal for high performance The file system is managed by the client Limited scalability 11

12 File Storage (NAS) TechTour Based on a traditional file system comprising files organized in hierarchical directories and subdirectories File level access to storage, as opposed to block level Storage exposed as a network file system NFS or CIFS/SMB Breaks down at large scale due to the overhead of the underlying file system that must be maintained The file system is managed by the storage server not the client NAS system has to manage user privileges 12

13 Object Storage TechTour Object storage, by contrast, doesn t split files up into raw blocks of data. Units of storage are called objects Objects can be grouped in buckets, but are otherwise stored in a flat address space Every object contains three things: The data itself Metadata A globally unique identifier (Applications identify the object via this ID) 13

14 Object Storage TechTour Good for unstructured data No real file system is needed here Unlimited scalability Geographic reach (access anywhere) Single namespace Built-in data resilience Extensible and flexible metadata tagging High throughput but doesn t perform well for low-latency, I/O intensive workloads Direct integration (via rest api) Gateway (allows for authentication) 14

15 Out with the old, in with the new? Not so fast- Both Block and NAS methods for storing data worked fine for years. So why is there a need for another concept, Object? Both Block and NAS need to implement functionality for user s access rights allowing them to make changes to the data. What we now see is that much of the data being archived is immured unstructured data. Content or materiel that will never be changed again. This is where Object storage shines.

16 Does it ever rain in the cloud? What if something goes wrong?

17 Protection: losing data is like death and taxes Parity-error correction calculated via an algorithm/raid-overhead as much as 50% Replication-creating multiple copies in your storage system Erasure Coding data is broken into fragments that are expanded and encoded with a configurable number of redundant pieces of data and stored across different locations. Efficient, usually around 20% overhead Recovery: With EC, the bigger the faster-the whole cluster is involved in the recovery and only recovers the data it needs RAID or NAS/SAN-bottlenecked through a single controller

18 Nine lives Drive fail probability 1/1000 EC is described as m:n, with m data and n parity So an EC of 10:2, 12 drives yields a probability of losing data (losing 3 drives) as one in a billion, so we have a durability of nine nines % Amazon s durability is eleven nines, or %

19 ACTIVE ARCHIVE GET THE RIGHT DATA IN THE RIGHT PLACE Lots of storage choices: Tape, Cloud, Object Store, Moving data from one type of storage to another while maintaining visibility Automatically optimizes where data is stored to reduce infrastructure costs and management overhead, minimizing silos and maximizing utilization Enhance business continuance through integrated backups, data replication, and multiple disaster recovery strategies Optimizes storage infrastructure and makes informed choices about expansion

20 One Ring to Rule Them All TechTour Asset Management Layer 20

21 Industry Trends Your data may be here, over there, up there or down here In addition to having a MAM or PAM control the movement of data, there will be new tools emerging this Fall from several manufacturers that are sophisticated Global Data Movers. DMs will cover everything from Ingest to Tiering, Migration and Archiving to any storage-disk- TAPE-WAN-CLOUD Index where the data is on any storage Smart Analytics to clarify storage spending, utilization and duplication Ability to search for assets POLICY BASED (not always a good thing), ability to quarantine files from migration

22 One Ring to Rule Them All TechTour DATA MANAGEMENT FABRIC-GDM 22

23 Industry Trends-Smarter Data Management Eliminate storage silos: globally manage all files across otherwise incompatible storage silos. Global Namespace creates a unified view across the entire storage environment simultaneously connecting all files regardless of vendor, platform and protocol. File migrations are automated across all storage with no disruption to users or applications. Intelligent data movement accelerates applications by ensuring files are always where they need to be, when they need to be there. User-defined policies ensure the hot and warm files are always on the right storage tier and cold data is on the lowest cost storage, on-premises or in the cloud. Data Insights delivers actionable intelligence for predictive, real-time analytics, and verification & reporting that facilitate storage planning and optimize application performance.

24 Industry Trends-Smarter Data Management Global queries across Extract, all storage auto-classification, tag, organize & manage Lifecycle management, QoS, & capacity optimization Automated policies based on schedule and triggers Data Insights & reporting Migrate, I/O balancing, tiering QUERY ENGINE ANALYTICS ENGINE VISUALIZATION ENGINE DYNAMIC DATA MOVER ENGINE METADATA ENGINE WORKFLOW ENGINE DISK TAPE WAN CLOUD

25 Metadata Driven Archives Image courtesy of Strongbox 25

26 Brand Convergence 26

27 REpresentational State Transfer 27

28

29 How do we move data into Object Storage?

30 30

31

32

33 San Francisco 49ers REST API Asset Management

34 Now deploying APIs for Google Cloud and Azure Video, Speech-to-text, Photo recognition, AI Rest API at AT&T Park for OnPrem Object Storage Game changer for Cutting Edge and our clients

35 Data over the WAN

36 Multi-Site Media Project Spring

37 Project Overview Goal: Increase collaboration between existing Stadium location and new office facility located 5 miles away in Oakland New site is linked to the Stadium by a high-bandwidth data circuit enabling multi-site workflows where editors can access media at either location Upgrade to Stadium Infrastructure included: New versions of Media Composer and Interplay Replace ISIS with new 240TB Avid Nexis Storage Add Transcoding (Content Agent) Improve Browsing (Avid MediaCentral UX) Upgraded Archive Add Jack London Square Systems: Add Edit Workstations and Browse/Edit Laptops Add More Avid Nexis Storage(240TB) Add Transcoding (Content Agent) Enable bi-directional workflow between Jack London Square & Stadium 38

38 10Gb Fibre 39

39 Workflow Overview Media Ingest from either location Interplay Check-In from either location Transcode to DNxHD 145 Browsing using Interplay or MediaCentral UX Editing Using media stored in either location Ability to begin edits at one site, finish at the other Archiving to XenData and Spectralogic T120 Ingest and transcode RED 4K media to Avid DNxHD 40

40 10Gb Fibre 41

41 Denver Broncos Multi-Site Solution Architecture Summer 2018

42 Deployment Plan Create an identical storage solution for both locations 2 x 96TB TerraBlock SAN Online Editorial Storage Pools Hub Server 192TB TerraBlock SAN Data Replication & Disaster Recovery Storage Repository Leverage existing technology and IT infrastructure at both locations Establish mirrored media transfers from location to location Use existing (non-dedicated) IT-supplied 10Gb circuit between the facilities 10Gb pipe will support mirroring of assets at night and weekends (avoiding IT bandwidth impacts) End Result: Editors at either location can access all media assets Automatic Disaster Recovery solution for both facilities 43

43 Benefits & Features Key Benefits Increased capacity Higher bandwidth Greater flexibility Disaster Recovery protection of Stadium and Practice Facility editorial assets 44

44 45

45 SF GIANTS-Cloud Editorial Demo MC Cloud, Central UX 46