Linux User Group of Davis. Marc J. Miller Strategic Alliance Manager, AMD October 7, 2003

Transcription

1 Linux User Group of Davis Marc J. Miller Strategic Alliance Manager, AMD October 7, 2003

2 x86 in High Performance Computing - The Six System Challenges #6: Watt density: x86 is the most widely installed instruction set in the world. With clusters exceeding 10,000 processors, watt #5: The I/O density is aninfrastructure: important issue. As cluster size Instruction expands, set cooling andto costs can be notcapacity relevant CPU significant. The bandwidth of a Front Side Bus causes an I/O performance ( to first order ). #4: Addressable memory: bottle-neck which continues to exclude IA32 from #3: Memory bandwidth: running Design the lowest watts/gig Cycle solution leveraging challenging parallel applications. What isstate-of-the-art important: AMD64 architecture and silicon-on Large RAM resident databases and memory insulator process With increased memory, come data intensive applications exceed the 4 gigabytes limit Provide a dedicated I/O system bus which is separate from theofmemory bus and keeps pace withstrides next generation intensive applications with and block sizes 32 bitcost systems. Paging (PAE) is not an #2: node: I/O protocols andper CPU processing clock. that cause cachefor thrashing. Making the cache larger acceptable solution performance reasons. is not cost effective. Hence, performance is limitedia32 Due to cost / performance and to I/Ox86-32: constraints, #1: Backward compatible AMD64 processing is the only solution clusters areon-chip limited to real twoand/or (2) processors, by the size of cache memory putting additional stress SMP cluster interconnect There bandwidth. is aon enormous investment is IA32 for all market segments. In many applications, porting code is not Bring 4 and 8 processor SMP systems closer in Improve an option. memory bandwidth and latency limit cost/performance to 2 processor systems; cache size $$$performance, decrease premium without Improve Provide a solution that is not only 100% backwardbreaking IA32 commodity economics; compatible, but designed to run IA32 code faster then Only possible - if the32-bit samearchitecture processor architecture any existing available. is used on the desktop. Provide a gradual and controlled migration path for porting to AMD64 Make the total cost of ownership minimal. 2

3 Advanced AMD Opteron Processor System Architecture AMD64 System DDR AMD Opteron Processor Separate Memory and I/O Paths Eliminates Most Bus C ontention Fewer C hips Needed For Basic Server IDE, FDC, USB, Etc. HyperTransport Buses for Glueless I/O or C PU Expansion Typical System Memory Access Delayed By Passing Through Northbridge HyperTransport Bus has ample Bandwidth for I/O Devices Bridge I/O Hub PCI Integrated memory controller Low latency memory access speeds processing Separate Memory and I/O pathways Eliminates I/O and memory bus competition Each processor has more memory & I/O paths Memory and I/O bandwidth scales well Modular glueless logic using HyperTransport technology bus Fewer chips and lower cost implementation Server Processor North Bridge I/O & Memory C ompete for C PU s FSB Bandwidth Bridge DDR More C hips Needed for Basic IDE, FDC, Server USB, Etc. Bandwidth Bottlenecks: Link B/W < I/O Device B/W South Bridge PCI Must access memory through Northbridge Longer latency memory access Memory and I/O access on the same bus I/O and Memory compete for bandwidth Memory or I/O paths originate from Northbridge Bandwidth does not scale well with more CPUs System logic uses more chips and many buses Systems cost more to design, build and test 3

4 AMD Opteron Processor MP System Architecture AMD Opteron System DDR DDR AMD Opteron AMD Opteron AMD Opteron AMD Opteron Typical MP System Processor DDR DDR DDR DDR Bridge Other I/O Other Bridge Bridge IDE, FDC, USB, Etc. I/O Hub PCI Processor Processor Memory Expander Memory Expander IDE, FDC, USB, Etc. North Bridge South Bridge Processor Bridge Bridge Bridge PCI Scalable memory and I/O bandwidth System scalability limited by Northbridge Up to 8 processors without glue logic Maximum of 4 processors o Processors compete for FSB bandwidth Each processor adds more memory Memory size and bandwidth are limited Each processor adds additional HyperTransport Maximum of 3 bridges technology buses for more and other I/O Many more chips required bridges Fewer chips required 4

5 Typical Multiprocessing System Typical MP System System scalability limited by Northbridge Max of 4 processors Processor Processor Processor Processor Processors compete for FSB bandwidth Memory size and bandwidth are limited of 3 bridges DDR Memory Expander Max Many more chips required DDR Memory Expander IDE, FDC, USB, Etc. North Bridge South Bridge Bridge Bridge Bridge PCI 5

6 Intel Xeon Light Load 6

7 Intel Xeon Heavy Load 7

8 AMD Opteron Processor 4P 800 Series Processor-based Server DDR AMD Opteron MHz 144-Bit Reg DDR cht DDR cht [1] [1] cht AMD Opteron cht [1] 133MHz AMD MHz HT [4] 66MHz AMD8131 Gbit Enet SCSI Gbit Enet DDR AMD Opteron IDE AC97 ENET USB AMD8131 [1] HT [3] HT [2] DDR AMD Opteron PCI-33 LPC Idle Latencies to First Data 1P System: <80ns 0-Hop in DP System: <80ns 0-Hop in 4P System: ~100ns 1-Hop in MP System: <115ns 2-Hop in MP System: <150ns 3-Hop in MP System: <190ns VGA FLSH BMC SIO [1] = 16x16 Coherent 1600MT/s [2] = 16x MT/s [3] = 8x8 400MT/s [4] = 8x8 1600MT/s 8

9 AMD Opteron Heavy Load 9

10 AMD Athlon 64 Processor Technical Overview AMD64 Technology An AMD64 PC can run both 32- and 64-bit operating systems START BOOT UP Using 32 bit BIOS Load 32 bit OS Run 32 bit Applications 32-bit Look at OS 64-bit Load 64 bit OS Run 32 & 64 bit apps 10

11 AMD Athlon 64 Processor Technical Overview AMD64 Technology 32-bit thread 64-bit thread 32-bit Application 64-bit Application THUNKING LAYER 64-bit Operating System 64-bit Device Drivers 11

12 AMD64 Technical Overview AMD s x86 Technology Family of Processors Compaq Alpha Sun SPARC x86 16 and 32 bits RISC & IA-64 HP PA-RISC (286, 386) MMX IBM Power 4 3DNow! Technology AMD64 Technology 32 and 64 bits A natural evolution of the current 32-bit architecture Similar to the 16- to 32-bit conversion of the 386 Designed to retain compatibility with the current installed base of x86 operating systems and applications Intel IA-64 (EPIC) There are many other 64-bit RISC solutions Each is a unique instruction set, all of which are incompatible with today s 32-bit code All require unique OS and applications Low-risk and low-cost path to high-performance computing 12

13 AMD64 Technology Building a Bridge from the 32- to the 64-bit World Leverages the initial success of AMD AthlonTM MP processor Adds 64-bit capabilities to the world s highest performing 32-bit core for 2P and 4P servers Current 32-bit applications will work on both 32-bit and 64bit operating systems Doesn t require special hardware or investment in a proprietary infrastructure Developing a solid ecosystem of motherboards, operating systems, development tools, and device drivers AMD64 32-bit Operating System 64-bit Operating System 32-bit Applications 32-bit Applications 64-bit Applications 13

14 AMD64 Computing Strategy (2) AMD64 Architecture: 64-bit integer registers In x86 64-bit Virtual Address Added by AMD64 52-bit Physical Address Sixteen 64-bit integer 127 XMM0 regs S S Sixteen 128-bit SSE regs E SSE2 Instruction Set Double precision scalar XMM7 and vector operations XMM8 16x8-, 8x16-way vector XMM8 packed integer operations SSE1 already added with AMD AthlonTM MP XMM15 Processor RAX 15 EAX 7 0 AH AL 0 79 G P R EAX 0 x 8 7 EDI R8 EIP R15 14

15 AMD64 Processor Overview Performance High-bandwidth integrated memory controller scales with processor frequency and number of processors L2 1MB Cache Compatibility Approximately 10,000 legacy applications at time of launch Scalability Can reduce costs for high-end systems Can remove I/O bottlenecks Easy multiprocessor scaling 16-bit HyperTransport technology links are at 1600MT/s; provides 6.4GB/s peak aggregate bandwidth AMD Opteron processor architecture Two DDR Memory Controllers L1 AMD64 Core Instruction Cache L2 Cache L1 Data Cache HyperTransport technology

16 AMD Opteron Processor Integrated Memory Controller Designed to run memory controller at processor speeds - not FSB speeds Designed to dramatically decrease latency DDR Memory Controller AMD64 Core L1 Instr Cache L1 Data Cache HyperTransport.... L2 Cache AMD Athlon processor 1P platforms achieve ~160 ns best-case latency AMD64 architecture is designed to achieve ~80 ns best-case latency Latency generally decreases further as the core frequency increases Designed to add intelligence without decreasing performance Designed to support multiple DDR memories DDR200, DDR266, and DDR333 Registered DIMMs Future processor cores planned to support DDR-II, etc. 16

17 AMD64 Processors And Target Systems AMD Opteron Processor 200 Series: AMD Athlon 64 Processor 2-way server & workstation processor Performance Desktop Processor 144-bit DDR interface per CPU: , 333 MHz Three 16-bit HyperTransport technology links per CPU. Typically, two are used to connect to another CPU and I/O 72-bit DDR interface 200, 266, 333, 400 MHz One 16-bit HyperTranport technology link 1 MB integrated L2 cache per CPU NOTE: The AMD Opteron Processor 800 Series: AMD Athlon 64 Up to 8-way server processor and 144-bit DDR interface per CPU: 200, 266, 333 MHz AMD Opteron Three 16-bit HyperTransport technology links per are processors CPU. Typically all three used to connect to other CPUs based on & I/O AMD64 1-MB integrated L2 cache technology 16-bit HyperTransport Links are at 1600MT/s; provides 6.4GB/s Peak Aggregate Bandwidth 17

18 AMD64 Benchmarks In test after test, AMD64 technology beats the competition 32-bit performance superior to other 32-bit solutions on the market 64-bit performance superior in terms of performance per dollar spent. In most cases more cost-effective to buy multiple AMD64 machines to get the same performance seen from a single competing 64-bit machine. See for the most recent data 18

19 Operating System Support Operating System Type SuSE Linux Enterprise Server (SLES) 8 32 & 64-bit SuSE Linux 9.0 Personal & Professional 32-bit & 64-bit UnitedLinux Version 1.0 code base by UnitedLinux Consortium 32 & 64-bit Conectiva Linux Enterprise Edition 32-bit Linux AMD64 kernel patches ( 64-bit Mandrake Linux 9.2 (coming soon) 32-bit & 64-bit Mandrake Linux Corporate Server bit & 64-bit NetBSD 32 & 64-bit Red Hat bit Red Hat Enterprise Linux 3 (coming soon) 32-bit & 64-bit Scyld Beowulf Cluster Operating System 32-bit Solaris 9 for x86 32-bit Turbolinux 8 for AMD64 32 & 64-bit Windows 2000 Server 32-bit Windows Server bit & 64-bit 19

20 AMD Athlon 64 Processor Technical Overview AMD64 Technology AMD64 Means Dynamic Scaling Large-scale simulations and games can be interacted with down to the lowest component level. From the largest view down to the smallest bolt, designers can maintain accurate physics at all times. 20

21 AMD Athlon 64 Processor Technical Overview AMD64 Technology AMD64 Means Large Memory Arrays Biometric Identification Photo-Realistic Gaming Instantaneous Access 21

22 AMD Athlon 64 Processor Technical Overview AMD64 Technology AMD64 Means Additional Registers Higher Level of Realism Real-Time Special Effects 22

23 AMD Athlon 64 Processor Technical Overview HyperTransport Technology Interface AMD Athlon 64 16x16 6.4GB/s Graphics Tunnel 8x8 800MB/s I/O Hub HyperTransport Technology Interface Attributes Unidirectional DDR-like performance (800MHz = 1600MT/sec) 4 bytes wide 6.4GB/sec bandwidth 23

24 AMD Athlon 64 Processor Technical Overview Reliability and Stability ECC Protection L1 data cache L2 tags and data Main memory DRAM (optional) Hardware Scrubbing Thermal Protection ThermTrip Shuts down processor without motherboard intervention Thermal Diode Works with motherboard circuitry to monitor CPU temperature and work with thermal control hardware (i.e, temp controlled fans, etc.) 24

25 AMD Athlon 64 Infrastructure Support Chipsets AMD Launch Chipsets Discrete Graphics Integrated Graphics I/O Hub AMD-8151 Graphics Tunnel 8x AGP VIA K8T400M+VT8235 AMD-8111 I/O Hub ATA 133, USB /100 Ethernet SiS x AGP, ATA 133 8x V-Link, USB /100 Ethernet 8x AGP, ATA 133 USB 2.0, 1394A VIA K8M400+VT8235 SiS x AGP + integrated gfx ATA 133, USB /100 Ethernet ALi 1563 ATA 133, USB /100 Ethernet 8x AGP + Ultra256 gfx ATA 133 USB 2.0, 1394A NVIDIA CrushK8 8x AGP, ATA 133 Two 10/100 Ethernet USB 2.0 NVIDIA CrushK8S 8x AGP, S-ATA, RAID Gigabit Ethernet USB 2.0, 1394A NVIDIA CrushK8G 8x AGP + GeForce4i gfx ATA 133, SATA Two 10/100 Ethernet USB 2.0, b Please contact the respective 3rd party vendors directly for latest schedules and information. 25

26 AMD Athlon 64 Infrastructure Support Chipset Interfaces AMD Athlon 64 16x16 6.4GB/s AMD x8 800MB/s Graphics Tunnel 754 PGA AMD 8111 I/O Hub or ALi 1563 I/O Hub AMD Athlon 64 16x16 6.4GB/s or 754 PGA AMD Athlon PGA VIA K8T400M SiS x16 6.4GB/s 533MB/s VIA VT GB/s SiS 963 I/O Hub or I/O Hub NVIDIA Crush K8 Platform Processor 26

27 HyperTransport Technology and Server Chipset Highlights

28 HyperTransportTM Technology Basics HyperTransportTM Technology buses have two unidirectional point-to-point links: The links can be 2-, 4-, 8-, 16-, or 32-bits wide in each direction HyperTransportTM links have a data rate up to 1.6 Gigabits/second per pin-pair (800 MHz clock) Total Aggregate Bandwidth = 12.8 Gbytes/second at 32 bits wide AMD OpteronTM supports three 16-bit HyperTransportTM links Provides 19.2 Gbytes/second on total data bandwidth 28

29 HyperTransportTM Technology Clock and Control Signals Ctrl Ctrl Clk Clk Asynchronous clock forwarding One clock is forwarded for each eight bits in each direction Clocks are double pumped; a 800 MHz clock is used for 1600 Mbit data rate Control line distinguishes command packets De-asserted during data packets In-band system management & legacy signal transport Eliminates sideband wires, interrupts use messages instead of wires Embedded code in back channel messages used for flow control Code indicates how many buffers are available for each virtual channel 29

30 HyperTransport Technology Data Structure Commands and interrupts are realized as a 32 bit command word Address and Data is preceded by a 64-bit header 6-bit type field Write, Read, Read Response, Fence & Flush 26-bit Command specific field 32-bit address field (command specific Byte or DWORD) Bit Time B it T im e Command Type Command Specific Information Read and Write Command Address D ata [ 7: 0] D ata [ 1 5: 8] D a ta [23:16] D a ta [31:24] D a ta [39:32] D a ta [47:40] D a ta [55:48] D a ta [63:56] 2 At 800MHz DDR it takes: 1.25ns to send a request (32-bits) 22.5ns to send a 64B block A write of one 64 byte block takes ~290ns + PCI X I/O latency 30

31 HyperTransport Technology Pin count Control Pair Clock Pair 8 Data Pairs Additional control signals Power OK (PWROK) Reset (RESET_L) Signal to ground ratio is conservatively 4:1 Optional link power down signals for mobile systems LDT_Stop DevReq Power per pin-pair is nil when a HyperTransport technology device is stopped (LDT_Stop) HT Device A HT Device B Clock Pair Control Pair 8 Data Pairs RESET_L PWROK V HT Gnd PWROK, RESET_L required for proper reset & init VHT routed between devices is required for proper common mode range Bus Width (Both Ways) Subtotal (high speed) VHT GND PWROK RESET_L Total Pins Data Pins (total) Clock Pins (total) Control Pins (total) DC Power per Pin-Pair: Signal to VLDT/Gnd Ratio: 4-9 mw, 6 mw Typical 4:1 31

32 HyperTransport Technology Intelligence Data movement over the HyperTransport bus does not use any CPU machine cycles. AMD External device can write to any address within the processor s physical 40-bit address range without CPU intervention. In cases where there are multiple HyperTransport technology ports, data can be passed between ports without CPU intervention. Because all devices reside within one physical 2^40 linear space all I/O devices have access to all processors and their associated memory & I/O. Opteron Opteron PGA PGA AMD Opteron Opteron PGA PGA 240 AMD Opteron Opteron PGA PGA AMD Opteron Opteron PGA PGA 32

33 HyperTransport Technology Scalable Bandwidth 15 HyperTransport Math: 8 Bits x 800 MHz x 2 transfers/clk x 2 (each direction) GBytes/Sec 8 Bits per Byte >2X = 3.2 GB/s Bi-Directional Throughput bits 64-bits 33Mhz PCI bits 64-bits 66Mhz PCI bits 133Mhz bits 4-bits 8-bits 16-bits 32-bits 800 MHz HyperTransport HT 2.0 (Planned) 33

34 AMD Opteron Chipset Roadmap AMD64 Technology = not POR AMD-8131 AMD 8132 HyperTransport HyperTransport Tunnel -2.0 Tunnel 2 Bridges 2 Bridges AMD-8111 HyperTransport HyperTransport I/O Hub I/O Hub AMD-8151 HyperTransport 8x AGP

35 HyperTransport Technology Building Blocks HyperTransport Link Mhz AMD-8131 HyperTransport Bridge Mhz Tunnel with 16-bit link to host (6.4 GB/s) & 8-bit to next device (3.2 GB/s) 2 independent channels designed to provide a peak of 1+1 GB/s of concurrent bandwidth HyperTransport Link Mhz AGP 8X Tunnel with 16 bits toward host (6.4 GB/s) & 8 bits to next device (3.2GB/s) AGP8X compatible 2 GB/s bandwidth AMD-8151 HyperTransport AGP Bridge HyperTransport Link AMD-8111 HyperTransport Southbridge 32 33Mhz PCI LPC FLASH Family of Bridges with up to 0.8 GB/s bandwidth State of the art I/O features Multiple solutions for different market segments SIO 35

36 HyperTransport Technology Future Building Blocks HyperTransport Link 533/ 266Mhz 2.0 AMD-8132 HyperTransport 2.0 Tunnel 533/ 266Mhz Tunnel with 16-bit host (8.0 GB/s) & 16-bit to next device (8.0 GB/s) Two independent 2.0 Ports 2.0 Supporting 533MHz, 266MHz, 133MHz, 100MHz, 66MHz, and Legacy-PCI modes I/O APIC HyperTransport Link 533/266Mhz 533/266Mhz HyperTransport Tunnel 16-bit host interface (8.0GB/s) 16-bit next device interface (8.0GB/s) Two independent 2.0 Ports Supporting 533MHz, 266MHz, 133MHz, 100MHz, 66MHz, and Legacy-PCI modes I/O APIC 36

37 HyperTransport Technology Consortium 37

38 AMD, the AMD Arrow logo, AMD Athlon, AMD Opteron, 3DNow! and combinations thereof, AMD-8111, AMD-8131, AMD-8132, and AMD-8151 are trademarks of Advanced Micro Devices, Inc. HyperTransport is a licensed trademark of the HyperTransport Technology Consortium. Microsoft and Windows are registered trademarks of Microsoft Corporation in the U.S. and/or other jurisdictions. Pentium and MMX are registered trademarks of Intel Corporation in the U.S. and/or other jurisdictions. SPEC and SPECfp are registered trademarks of Standard Performance Evaluation Corporation in the U.S. and/or other jurisdictions. Alpha is a trademark of Digital Equipment Corporation. MIPS is a registered trademark of MIPS Technologies, Inc. Other product and company names used in this presentation are for identification purposes only and may be trademarks of their respective companies. 38