Omneon MediaGrid Technical Overview
Introduction To MediaGrid Property Architecture Scalability Networking Client Access High Availability Application Integration Attributes Scale-out; MediaFS clustered file system 12TB to petabytes 375MB/s to tens of GB/s Linear, non-disruptive scaling Ethernet Both 10GBe and 1GbE File System Driver (FSD) NAS protocols (CIFS, FTP, NFS, AFP) No single point of failure Online maintenance Transparent recovery from hardware failures Tested and optimized with dozens of leading media applications
MediaGrid Storage System Architecture and Key Components ü Clustered File System ContentDirectors File System Driver Ethernet ContentServers File Slice Storage File Slicing/Assembly File System Metadata
File System Driver Parallelizing Client Access FSD: a key enabler of the MediaGrid performance advantage Slicing/ Assembly File FSD For: Parallel access to slices Slices
File System Driver Maximizing Performance Variable slice size File Type.ts.mov.aiff.xyz Slice Size 8MB 2MB 512K 256K ü Can be configured based on file type and folder Intelligent Caching FSD Requested data (slice N) Slice N+1 Slice N+2 Slice N+3 Slice N+4 Slice N+5 Slice N+6 ü Use client memory as a cache ü Great for sequential access Optimizes performance for both small and large files! Often enables performance levels of local storage!
MediaGrid I/O Flow File System Directory ContentDirectors File Slicing/Assembly Slice Server Data Read/Write FSD File create/open/close Server IP map File slice read/write Ethernet ContentServers RAID Groups
Other Clustered NAS Bottleneck prone and High Latency File System Meta Data Slice Storage File Slicing/Assembly Ethernet Storage Nodes Client NFS/CIFS NAS protocols don t leverage distributed file systems
File System Driver Superior Single Client Performance ContentDirectors File System Driver Ethernet ContentServers High bandwidth for a single client (>1GBs)) Consolidate to fewer and powerful servers (lower operational cost) Speed up workflows (ingest more streams, more transcode jobs, use uncompressed ) File Slice Storage File System Metadata File Slicing/Assembly
File System Driver Add Clients Without Impacting Performance File System Driver ContentDirectors File System Driver Ethernet ContentServers File System Driver File Slice Storage File System Metadata File Slicing/Assembly Great aggregate bandwidth for multiple clients Add servers without worrying about performance impact Consistent latency with increasing client count
File System Driver No Fragmentation Issues as system fills up ContentDirectors File System Driver Ethernet ContentServers File Slice Storage File System Metadata File Slicing/Assembly Consistent performance at 10% full or 90% full No periodic defragmentation needed Improves operational efficiency
MediaGrid: Simple Ethernet Infrastructure Clients ü One network LAN ü Simple, standard Ethernet ü Integrated MediaFS file system MediaGrid Storage ü Single vendor solution
FC SANs: Complex And Expensive StorNext LAN Clients StorNext Clustered LAN Gateway StorNext Metadata controller ü Two networks $$ ü FC SAN HBAs $$ LAN SAN ü HBA compatibility headaches ü Need gateway if you cannot use HBA ü Different vendors for file system and storage StorNext SAN Clients SAN Storage
System Components ContentDirectors ü Deployed in failover pair ü 1U each ContentServers ü Dual controllers ü 1GbE or 10GbE ü 16 x 1, 2, or 3TB SATA ü Hot-swappable, 3U ContentStores (Cost-effective Scaling!) ü JBODs ü 0-5 per ContentServer ü 16 x 1, 2, or 3TB SATA ü Hot-swappable, 3U
Protection Against Drive Failures Advanced software RAID Proven in 10 years of use with nonstop video servers Two RAID options RAID 4 (2+1) for maximum performance RAID 6 (6+2) for balanced capacity and performance Hot-swap failed drives Spare drive available for RAID 4 Application I/O prioritized over rebuild I/O Minimize performance impact during rebuilds
ContentServer Built in High Availability No single point of failure! ü Dual Hot-pluggable Power Supplies ü Dual active-active controllers ü Transparent failover
Campus-Level Disaster Recovery Synchronous Replications Under same file Replicate across two groups of enclosures, physically separated but in a single file system Groups can be up to 500M apart* Active-active configuration Location 1 Location 2 Benefit: Instant, transparent failover upon site failures (fire, flood, power outage)! Automatic failback and replication on recovery Single File System, Replicated data Group A Group B
Thank You!
Optimized for Media File System tuned for media files and I/O For Sequential access Qualified with dozens of media applications Using the File System Driver Performance tuning and sizing Add on Media Workflow Software Media Application Server (Media Management and Automation) MXFServer (Project Sharing and Collaborative Editing)
Qualified With Dozens Of Media Applications Ingest Ingest Spectrum/ MediaDeck ProMedia Capture Digital Rapids Amberfin icr MOG SpeedRail EVS Editing Editing Apple Final Cut Pro Avid Media Composer Adobe Premiere Grass Valley Edius Autodesk Smoke MXFServer Transcoding Transcoding ProMedia Carbon Amberfin Telestream Harmonic QC QC ProMedia QC Interra Baton Archival Archival Atempo Front Porch Digital SGL Xendata Media Management Media Management Media Application Server MXF Server Ardendo Dalet Origin Serving Origin Serving ProMedia Origin Server Microsoft IIS Adobe FMS Partner
Edit Performance Industry-leading Edit-In-Place Seat Count Torture Timeline Edit Seat Count Testing Edit playback performance/scalability with stringent timelines Results using FCP, but applicable to Avid and Adobe as well 1x ContentServer 1xContentServer+1xContentStore Streams per Seat RAID 6 RAID 4 RAID 6 RAID 4 2x50Mb Streams 15 20 30 40 2x100Mb Streams 7 9 17 23 2x220Mb Streams 5 6 9 13 2x330Mb Streams 4 5 8 10
AutoDesk Smoke on MediaGrid Smoke customers finally have an alternative to SANs! 2K/4K Editing 10GbE clients Support for both uncompressed DPX and QT formats 21
MXFserver Enabling Collaboration with Control Share project data between Editors Avid MediaComposer, Apple Final Cut Pro and Adobe Premiere MediaComposer style bin locking Control access to media Restricts user access to authorized projects only Prevent unauthorized users from corrupting, deleting source media files Allows user s rights to be managed Allow browsing but not editing content Restricts those who may delete media
Faster Transcoding With MediaGrid Promedia Carbon performance on MediaGrid: Up to 2.5X Isilon performance! Relative Performance: MediaGrid vs. Isilon *Both systems had same number of SATA drives
Optimized for Multiscreen Origin Storage MediaGrid optimized for Smooth and HLS Streaming Qualified with Harmonic ProMedia Origin and Microsoft IIS Consolidate to high performance shared storage Server hundreds of streams per Origin Server MediaGrid
Media Application Server (MAS) Adding Media Intelligence to File System MAS is a Media Management Software Appliance that works with MediaGrid, Servers and 3 rd Party storage Provides media awareness to files Harness existing clip meta-data and add user meta-data Organize and search content Manage file system operations Purge old content Mirror/Move files Trigger workflow tasks via API Transcode (Rhozet) QC (Rhozet, Interra Baton) CC Extraction Archival (XenData, Quantum) Proxy Creation (IPV) 25
MediaGrid Efficient Scaling Ethernet ContentDirectors ContentServer 4 Quad core CPU s 24GB Memory Up to 40Gb/s networking 16 Internal Drives
MediaGrid Efficient Scaling Ethernet ContentDirectors ContentServer 4 Quad core CPU s 24GB Memory Up to 40Gb/s networking 16 Internal Drives ContentStore 16 internal drives each up to 5 ContentStores per ContentServer 2x24Gb SAS2
MediaGrid Efficient Scaling ContentDirectors Ethernet ContentServers ContentStores Continue adding to linearly scale capacity and BW
MediaGrid Key Components Storage Nodes ü Building blocks for scaling ContentServer ContentStore Includes Storage and processing Storage only Rack Units 3U 3U Ethernet Storage Reliability 12 x 1Gbe or 4 x 10Gbe 16 x SATA2 Drives (1TB, 2TB, 3TB) Dual Controllers Dual Power supplies 16 x SATA2 Drives (1TB, 2TB, 3TB) Dual Controllers Dual Power supplies
ContentDirectors ü Stores file system structure (journaled) ü Serves file metadata Best For ContentDirector File count <1M, <500TB Rack Units 1U 1U High Performance ContentDirector File count >1M, >500TB Ethernet 4 x 1GbE 4 x 1GbE Storage 2 x 250GB SATA 2 x 1TB SATA and 300GB SSD RAM 12GB 32GB High Availability Redundant power supplies Redundant power supplies
ContentBridge Access via NAS Protocols ContentDirectors ücifs ünfs üftp ContentBridge ContentServers Ethernet FSD Ethernet ContentBridges for access via NAS protocols Scale protocol layer separately from backend storage
ContentBridge Specs For NFSv3, CIFS, and FTP Access ContentBridge High Bandwidth ContentBridge Rack Units 1U 1U Ethernet 2 x 1GbE 1 x 10GbE RAM 4GB 12GB High Availability Redundant power supplies Redundant power supplies