Intel Xeon Phi TM Coprocessor Architecture and Tools

Size: px

Start display at page:

Download "Intel Xeon Phi TM Coprocessor Architecture and Tools"

Elinor Hicks
5 years ago
Views:

1 Intel Xeon Phi TM Coprocessor Architecture and Tools The Guide for Application Developers Rezaur Rahman

2 Intel Xeon Phi Coprocessor Architecture and Tools: The Guide for Application Developers Rezaur Rahman Copyright 2013 by Apress Media, LLC, all rights reserved ApressOpen Rights: You have the right to copy, use and distribute this Work in its entirety, electronically without modification, for non-commercial purposes only. However, you have the additional right to use or alter any source code in this Work for any commercial or non-commercial purpose which must be accompanied by the licenses in (2) and (3) below to distribute the source code for instances of greater than 5 lines of code. Licenses (1), (2) and (3) below and the intervening text must be provided in any use of the text of the Work and fully describes the license granted herein to the Work. (1) License for Distribution of the Work: This Work is copyrighted by Apress Media, LLC, all rights reserved. Use of this Work other than as provided for in this license is prohibited. By exercising any of the rights herein, you are accepting the terms of this license. You have the non-exclusive right to copy, use and distribute this English language Work in its entirety, electronically without modification except for those modifications necessary for formatting on specific devices, for all non-commercial purposes, in all media and formats known now or hereafter. While the advice and information in this Work are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. If your distribution is solely Apress source code or uses Apress source code intact, the following licenses (2) and (3) must accompany the source code. If your use is an adaptation of the source code provided by Apress in this Work, then you must use only license (3). (2) License for Direct Reproduction of Apress Source Code: This source code, from TouchDevelop: Programming on the Go, ISBN is copyrighted by Apress Media, LLC, all rights reserved. Any direct reproduction of this Apress source code is permitted but must contain this license. The following license must be provided for any use of the source code from this product of greater than 5 lines wherein the code is adapted or altered from its original Apress form. This Apress code is presented AS IS and Apress makes no claims to, representations or warrantees as to the function, usability, accuracy or usefulness of this code. (3) License for Distribution of Adaptation of Apress Source Code: Po:rtions of the source code provided are used or adapted from TouchDevelop: Programming on the Go, ISBN copyright Apress Media LLC. Any use or reuse of this Apress source code must contain this License. This Apress code is made available at Apress.com/ as is and Apress makes no claims to, representations or warrantees as to the function, usability, accuracy or usefulness of this code. ISBN-13 (pbk): ISBN-13 (electronic): Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. President and Publisher: Paul Manning Lead Editors: Jeffrey Pepper (Apress); Patrick Hauke (Intel) Development Editor: Robert Hutchinson Coordinating Editor: Anamika Panchoo Cover Designer: Anna Ishchenko Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY Phone SPRINGER, fax (201) , orders-ny@springer-sbm.com, or visit For information on translations, please rights@apress.com, or visit

3 About ApressOpen What Is ApressOpen? ApressOpen is an open access book program that publishes high-quality technical and business information. ApressOpen ebooks are available for global, free, noncommercial use. ApressOpen ebooks are available in PDF, epub, and Mobi formats. The user friendly ApressOpen free ebook license is presented on the copyright page of this book.

4 To my mother, who has always been so proud of me and wanted for me to be my best, and to my mother-in-law, who has eagerly been waiting for this book to be published

5 Contents at a Glance About the Author... xv About the Technical Reviewer... xvii Acknowledgments... xix Introduction... xxi Part 1: Hardware Foundation: Intel Xeon Phi Architecture... 1 Chapter 1: Introduction to Xeon Phi Architecture...3 Chapter 2: Programming Xeon Phi...15 Chapter 3: Xeon Phi Vector Architecture and Instruction Set...31 Chapter 4: Xeon Phi Core Microarchitecture...49 Chapter 5: Xeon Phi Cache and Memory Subsystem...65 Chapter 6: Xeon Phi PCIe Bus Data Transfer and Power Management...81 Part 2: Software Foundation: Intel Xeon Phi System Software and Tools...95 Chapter 7: Xeon Phi System Software...97 Chapter 8: Xeon Phi Application Development Tools Part 3: Applications: Technical Computing Software Development on Intel Xeon Phi Chapter 9: Xeon Phi Application Design and Implementation Considerations Chapter 10: Application Performance Tuning on Xeon Phi v

6 Contents at a Glance Chapter 11: Algorithm and Data Structures for Xeon Phi Chapter 12: Xeon Phi Application Development on Windows OS Appendix A: OpenCL on Xeon Phi Appendix B: Virtual Shared Memory Programming on Xeon Phi Index vi

7 Contents About the Author... xv About the Technical Reviewer... xvii Acknowledgments... xix Introduction... xxi Part 1: Hardware Foundation: Intel Xeon Phi Architecture... 1 Chapter 1: Introduction to Xeon Phi Architecture...3 History of Intel Xeon Phi Development...4 Evolution from Von Neumann Architecture to Cache Subsystem Architecture... 5 Improvements in the Core and Memory... 6 Interconnect and Cache Improvements Intel Xeon Phi Coprocessor Chip Architecture...12 Applicability of the Intel Xeon Phi Coprocessor...13 Summary...14 Chapter 2: Programming Xeon Phi...15 Intel Xeon Phi Execution Models...15 Development Tools for Intel Xeon Phi Architecture...16 Intel Composer XE Setting Up an Intel Xeon Phi System...19 Install the MPSS Stack Install the Development Tools Code Generation for Intel Xeon Phi Architecture...20 Native Execution Mode vii

8 Contents Language Extensions to Support Offload Computation on Intel Xeon Phi...22 Heterogeneous Computing Model and Offload Pragmas Language Extensions and Execution Model Runtime Library Routines Offload Example Summary...30 Chapter 3: Xeon Phi Vector Architecture and Instruction Set...31 Xeon Phi Vector Microarchitecture...31 The VPU Pipeline Vector Registers Vector Mask Registers Extended Math Unit Xeon Phi Vector Instruction Set Architecture...36 Data Types Vector Nomenclature Vector Instruction Syntax Xeon Phi Vector ISA by Categories Summary...47 Chapter 4: Xeon Phi Core Microarchitecture...49 Intel Xeon Phi Cores...49 Core Pipeline Stages...50 Cache and TLB Structure...52 L2 Cache Structure...55 Multithreading...55 Performance Considerations Probing the Core Summary...64 viii

9 Contents Chapter 5: Xeon Phi Cache and Memory Subsystem...65 The Interconnect Topologies for Manycore Processors...65 Bidirectional Ring Topology Two-Dimensional Mesh Topology Two-Dimensional Torus Topology Other Topologies The Ring Interconnect Architecture in Intel Xeon Phi...67 L2 Cache...68 Tag Directory Data Transactions The Cache Coherency Protocol Hardware Prefetcher Memory Transactions Flow...73 Cacheable Memory Read Transaction Managing Cache Hierarchy in Software Probing the Memory Subsystem...77 Measuring the Memory Bandwidth on Intel Xeon Phi Summary...80 Chapter 6: Xeon Phi PCIe Bus Data Transfer and Power Management...81 DMA Engine...83 Measuring the Data Transfer Bandwidth over the PCIe Bus Reading Data from the Coprocessor...87 Low-Level Data Transfer APIs for Intel Xeon Phi...88 Placement of PCIe Cards for Optimal Data Transfer BW...90 Power Management and Reliability...90 Idle Stare Management Reliability Availability and Serviceability Features in the Intel Xeon Phi Coprocessor Summary...93 ix

10 Contents Part 2: Software Foundation: Intel Xeon Phi System Software and Tools Chapter 7: Xeon Phi System Software...97 System Software Component...98 Ring 0 Driver Layer Components of the MPSS...99 System Boot Process Coprocessor OS Creating a Third-Party Coprocessor OS mic0: Transition from State Booting to Online Host Driver Linux Virtual File System (Sysfs and Procfs) Networking on Xeon Phi Network File System Open Fabrics Enterprise Distribution and Message Passing Interface Support System Software Application Components Summary Chapter 8: Xeon Phi Application Development Tools The Application Development Tools Intel C/C++ Composer XE OpenMP 4.0 and Language Extensions Pragmas Asynchronous Data Transfer Over PCI Express Keywords Using Shared Virtual Memory Valid Use of the Keywords Macros Intrinsics C++ Class Libraries Application Programming Interfaces Environment Variables Compiler Options Creating Offload Libraries x

11 Contents Intel Fortran Composer XE Directives Macros Application Programming Interfaces Environment Variables, Compiler Options, and Creating Static Libraries Third-Party Compilers Supporting Xeon Phi CAPS Compiler Debugging Xeon Phi Applications Intel Debugger Third-Party Debuggers Optimization Tool: Intel Vtune Amplifier XE Libraries Native or Symmetric Execution Compiler-Assisted Offload Using the Automatic Offload Version of the MKL Library Third-Party Math Libraries Intel Cluster Tools Third-Party Cluster Tools Summary Part 3: Applications: Technical Computing Software Development on Intel Xeon Phi Chapter 9: Xeon Phi Application Design and Implementation Considerations Workload-Related Considerations Gustafson s Law Scaled Speedup Effect of Grid Shape on Performance Algorithm Considerations Data Structure xi

12 Contents Offload Overhead Load Balancing Implementation Considerations Memory Management Mixed-Precision Arithmetic Optimizing Memory Transfer Bandwidth over the PCIe Bus Data Alignment Considerations Communication Summary Chapter 10: Application Performance Tuning on Xeon Phi Getting Baseline Data Timing Applications Detecting Application Execution Bottlenecks Some Basic Performance Events Setting Target Performance Optimizing Code Compiler-Driven Optimizations Data Alignment Removing Pointer Aliasing Streaming Store Using Large Pages Using Intel Cilk Plus Array Notation Vectorization with Intel Compiler Using the Math Kernel Library Cluster-Level Tuning Summary xii

13 Contents Chapter 11: Algorithm and Data Structures for Xeon Phi Algorithm and Data Structure Design Rules for Xeon Phi General Matrix-Matrix Multiply Algorithm (GEMM) Rules 1 and 3: Scalable Parallelization and Optimal Cache Reuse Rule 2: Efficient Vectorization Molecular Dynamics Rule 1: Scalable Parallelization Rules 2 and 3: Efficient Vectorization and Optimal Cache Reuse Stencil Operation Rule 1: Scalable Parallelization Rule 2: Efficient Vectorization Rule 3: Optimal Cache Reuse European Option Pricing Using Monte Carlo Simulation in Financial Applications Rule 1: Scalable Parallelization Rule 2: Efficient Vectorization Rule 3: Optimal Cache Reuse Summary Chapter 12: Xeon Phi Application Development on Windows OS MPSS MPSS Tools Development Tools Language Extensions for the Xeon Phi Coprocessor Offload Environment Variables Debugging Offload Execution Logging into Xeon Phi Console using PuTTY Using VTune Amplifier XE to Profile Offload Code on Windows Building and Running Xeon Phi Native Applications from the Windows Host Summary xiii

14 Contents Appendix A: OpenCL on Xeon Phi Installation Building and Running OpenCL Application Performance Optimization Appendix B: Virtual Shared Memory Programming on Xeon Phi Placing Data on the Virtual Shared Memory Region Shared Functions Synchronizing Between the Host and the Coprocessors Index xiv

15 About the Author Rezaur Rahman is a Senior Software Engineer in the Intel Software and Services Group. He played a key role in the inception and development of the Xeon Phi coprocessor for technical computing applications by demonstrating the viability of applying Intel s manycore graphics processor codenamed Larrabee to solving technical computing problems. He led the worldwide technical enabling team for Intel Xeon Phi products, focused on porting and optimizing applications on the Xeon Phi coprocessor for hundreds of technical computing customers. He has worked internally with hardware architects and Intel compiler and tools teams to optimize and add features to improve the performance of Intel Many Integrated Core (MIC) and Xeon Phi software and hardware components. With 25 years experience in computer architecture and software design, Rahman contributes his expertise in technical code optimization, performance tuning, and processor microarchitectural analysis in the HPC domain. Rahman holds a master s degree in computer science from Texas A&M University and a bachelor s in electrical engineering from Bangladesh University of Engineering and Technology. xv

17 About the Technical Reviewer Leonardo Borges is a Senior Staff Engineer in the Intel Software and Services Group. He realizes system and software design and optimization with a primary focus on the energy vertical. Borges has specialized in applying his background in numerical analysis and parallel numerical mathematics libraries development in the HPC domain for the past two decades. He joined the Intel Many Integrated Core (MIC) program at its early stages. Borges holds a master s degree in applied mathematics and a PhD in computer science from Texas A&M. xvii

19 Acknowledgments I appreciate all the support and encouragement I received while writing the book from my wife Farzana, my daughters Fariha and Ridwana, and my father. My sincere thanks go to the ApressOpen publishing team Robert Hutchinson, Anamika Panchoo, and others and to the technical reviewer, Leo Borges, for helping me get through the tough job of completing the book by editing the presentation and helping to refine the content. Thanks to Patrick Hauke and Stuart Douglas from Intel Press for encouraging me to take on the project and providing the necessary support to get it published. Finally, I appreciate the unstinting help from my engineering colleagues at Intel who have helped me understand and improve my knowledge of the architecture and optimization techniques for the Intel Xeon Phi coprocessor. xix

21 Introduction This book provides a comprehensive introduction to Intel Xeon Phi architecture and the tools necessary for software engineers and scientists to develop optimized code for systems using Intel Xeon Phi coprocessors. It presents the in-depth knowledge of the Xeon Phi coprocessor architecture that developers need to have to utilize the power of Xeon Phi. My book presupposes prior knowledge of modern cache-based processor architecture, but it begins with a review of the general architectural history, concepts, and nomenclature that I assume my readers bring. Because this book is intended for practitioners rather than theoreticians, I have filled it with code examples chosen to illuminate features of Xeon Phi architecture in the light of code optimization. The book is divided into three parts corresponding to the areas engineers and scientists need to know to develop and optimize code on Xeon Phi for high-performance technical computing: Part 1 Hardware Foundation: Intel Xeon Phi Architecture sketches the salient features of modern cache-based architecture with reference to some of the history behind the development of Xeon Phi architecture that I was personally engaged in. It then walks the reader through the functional details of Xeon Phi architecture, using code samples to disclose the performance metrics and behavioral characteristics of the processor. Part 2 Software Foundation: Intel Xeon Phi System Software and Tools describes the system software and tools necessary to build and run applications on the Xeon Phi system. I drill into the details of the software layers involved in coordinating communication and computations between the host processor and a Xeon Phi coprocessor. Part 3 Applications: Technical Computing Software Development on Intel Xeon Phi discusses the characteristics of algorithms and data structures that are well tuned for the Xeon Phi coprocessor. I use C-like pseudo-algorithms to illustrate most instructively the various kinds of algorithms that are optimized for the Xeon Phi coprocessor. Although this final part of the book makes no pretensions to being comprehensive, it is rich with practical pointers for developing and optimizing your own code on the Xeon Phi coprocessor. Although each of the three parts of the book is relatively self-contained, allowing readers to go directly to the topics that are of most interest to them, I strongly recommend that you read Part 1 for the architectural foundation to understand the discussion of algorithms in Part 3. These algorithms are mainly of practical interest to the Xeon Phi community for optimizing their code for this architecture. xxi

Windows 10 Revealed. The Universal Windows Operating System for PC, Tablets, and Windows Phone. Kinnary Jangla

Windows 10 Revealed. The Universal Windows Operating System for PC, Tablets, and Windows Phone. Kinnary Jangla Windows 10 Revealed The Universal Windows Operating System for PC, Tablets, and Windows Phone Kinnary Jangla Windows 10 Revealed Kinnary Jangla Bing Maps San Francisco, California, USA ISBN-13 (pbk): 978-1-4842-0687-4