Benchmarking Porting Costs of the SKYNET High Performance Signal Processing Middleware

Transcription

1 Benchmarking Porting Costs of the SKYNET High Performance Signal Processing Middleware Michael J. Linnig and Gary R. Suder Engineering Fellows Sept 12, 2014

2 Agenda The Need for Speed High Performance Parallel Processing in Airborne Radar Overview of SKYNET Middleware Approach History of SKYNET Middleware Grid Based Computing with SKYNET Measuring Portability Costs 2

3 Motivation for Skynet Signal Processing Middleware Multiple simultaneous receive beams. Prodigious Data Rates 100 s of millions of samples per second from each receive beam Tens of gigabytes per second Need Many Tens of TeraFLOPS 9/5/ Multiple Processor Software Architecture Needed

4 Single Program Multiple Data (SPMD) Programming Paradigm Decompose Computations Radar is an Embarrassingly Parallel Problem Each processor assigned a subset of the data Allocate to a Large Number of Processors Running in Parallel Each Processor Working On A Subset Of The Data Assigned subset may be based on geographic location, or other convenient scheme SKYNET Supports Multiple SPMD applications linked together

5 SKYNET Middleware A Portable Open Architecture for Massively Parallel Computing Solves an Embarrassingly Parallel Problem Open Architecture based on FOSS Supercomputer Standards 10 years+ on Wide Area Surveillance Radars. Nicknamed SKYNET with a nod to James Cameron Single Program Multiple Data (SPMD) Approach Abstraction is a Rectangular Grid of processors Based on decades of high-performance computing experience Grid abstraction is inherently scalable to a variable number of processors Supports compute power needed for Radar Radar Data Parallel Array of Processors Users include Raytheon, Northrop Grumman and five other companies Allows Hundreds of Processors to Be Netted Together

6 History of Skynet Signal Processing Middleware The originally on an Array Of Hundreds of COTS Digital Signal Processors Array Operating System (AOS) portion of the SKYNET middleware added grid-based inter-processor communication between applications. The SKYNET Middleware API isolates radar modes from the details of the underlying Operating System and Processor Modern Platforms In 2003, Northrop Grumman Corporation revised SKYNET middleware to use MPI protocol internally Raytheon Optimized the SKYNET Middleware on Linux Raytheon easily ported existing radar modes to an array of Linux based COTS high performance processors by using the SKYNET Middleware. Linux Open MPI 9/5/ Ported to six different architectures, including POWER and Intel

7 SKYNET Processor Middleware Approach Embrace COTS Signal Processors Use Middleware to isolate radar applications Built on Linux OS, to allow for easy migration to new processors Uses Widely Used Open Standards No Proprietary Interfaces Decomposed into two parts: Government owned SKYNET MetaWare a thin supercomputing framework that decouples radar applications from operating system and hardware dependency. Open Middleware A open compute environment, based on Linux and open standards, can be easily targeted to new hardware platforms. Uses Open Standards, Linux, DDS, POSIX Middleware Solution Features: Entire software stack is either Government owned or open standards High Performance Computing needed for Wide Area Surveillance Easy upgrades, multiple vendors of Linux processors Foundation of the SKYNET Middleware is Linux 7 SKYNET Enables Independent Procurement of Processor and Modes

8 The SKYNET Virtual Node Abstraction Communicating Virtual Nodes Middleware provides an abstraction called a Virtual Node, that is layered upon the real-time OS abstraction and the Message Passing Interface abstraction. A Virtual Node is a task that is defined to have four NEWS neighbors (North, East, West and South, hence NEWS.). This approach allows efficient algorithms spanning hundreds of processors. The resulting radar mode software is largely independent of the number and the kind of processors. Middleware abstraction hides all detail of communication Takes advantage of Multicore and Multiple CPU Processors Natural growth path to using processors with hundreds of cores on a single chip NEWS -- Cartesian Communication Optimized Other SKYNET Functions Semaphores Queues Mailboxes Timers Memory Management Task Management Periodic Alarms Logging Middleware Provides Operations that Operate Across Processor Boundaries Transparently

9 Benchmarking Middleware Portability Two Operational Radar Modes ported to Eight Different Vendor Platforms Modes had to meet existing execution requirements Changes to Modes SW documented Middleware 9/5/2014 Enabled Port of Modes to Most Platforms with no changes 9

10 Summary The SKYNET Middleware is an example Open Systems Architecture that allows rapid technology refresh A Signal Processing Middleware can be Open and Portable While Maintaining High Performance A Signal 9/5/2014Processing Middleware can be Portable and Efficient 10