Industrial-Strength High-Performance RISC-V Processors for Energy-Efficient Computing Dave Ditzel dave@esperanto.ai President and CEO Esperanto Technologies, Inc. 7 th RISC-V Workshop November 28, 2017 November 28, 2017 Esperanto Technologies Inc. 1
Esperanto Presenters at 7 th RISC-V Workshop Three presentations from Esperanto people... would have made sense to do in the order below: Chris Celio from U.C. Berkeley will discuss BOOM v2 Wednesday at 9:36am Chris has joined Esperanto... so what happens to BOOM? Esperanto will continue to manage and support the open-source BOOM repository. Expect BOOM repository to move over to Free Chips Project. Esperanto plans to implement even higher performance out-of-order processors, will discuss that a bit today. Roger Espasa from Esperanto will discuss progress on RISC-V vector extensions Wednesday at 1:30pm Roger and Krste are co-leading the vector ISA proposal. Esperanto is doing a real implementation to understand design tradeoffs and gain feedback to improve the proposal. Plan to use RISC-V vector ISA in Esperanto products. Dave Ditzel from Esperanto Tuesday 11:36am What are all these Esperanto people doing? Today is for exiting stealth mode and providing project status to the RISC-V community, not a product announcement. Will share our vision for expanding the RISC-V market, and we want your help and to hear your feedback. Caveat discussion is about work in progress and subject to change. November 28, 2017 Esperanto Technologies Inc. 2
Strengthening perception of RISC-V in the industry RISC-V is off to a great start Captured university interest and new architecture research. Chisel and other CAD innovations provide many benefits in rapid design. RISC-V Rocket has launched many projects and company products. Several companies offering RISC-V designs and IP. RISC-V is doing well at gathering mindshare for embedded applications. But still many in industry view RISC-V as a curiosity or toy, only for low end. We repeatedly heard the following questions: Where is the RISC-V alternative to high-end ARM processors? Can we get a high-end RISC-V design with human-readable synthesizable Verilog? When will we see RISC-V with compelling high-end performance in leading edge process like 7nm? CPU is only a small part of SoC, what do we do for graphics if we use RISC-V? Where are the RISC-V processors for machine learning? Where are the RISC-V processors for HPC? November 28, 2017 Esperanto Technologies Inc. 3
Expanding RISC-V s piece of the pie Intel x86 AMD Andes RISC-V November 28, 2017 Esperanto Technologies Inc. 4
Esperanto: Expanding RISC-V s piece of the pie Esperanto is a startup company with a passion to help RISC-V succeed. We want to complement the many good companies like SiFive, Andes, etc. to build the ecosystem. Most of us have backgrounds in high-performance and low-power processor design. High performance is where Esperanto can have most impact on the RISC-V ecosystem. Esperanto is designing a high-performance RISC-V core comparable to the best IP alternatives. Esperanto is designing an energy-efficient RISC-V core for high TeraFLOP computing needs. Goal is to make RISC-V MORE COMPELLING than the other high-end alternatives. November 28, 2017 Esperanto Technologies Inc. 5
Industrial Strength RISC-V Processors To be compelling Esperanto is Designing our RISC-V processors in leading edge 7nm CMOS. Designing for both highest single thread performance and best TeraFLOPS/Watt. Using methodologies comfortable for large IP customers. Mainstream CAD tools Customers asking Esperanto for RISC-V IP with human readable synthesizable Verilog More familiarity with Verilog Easier to maintain and modify with their existing engineering talent Common commercial CAD tool support Strong physical design effort 7nm scaling challenges make physical design increasingly important. Energy efficiency needs careful tradeoff of architecture, circuits and physical design. Flagship for RISC-V Want to build a product that will draw attention to great RISC-V performance. So what are we building? November 28, 2017 Esperanto Technologies Inc. 6
Esperanto s Mission Esperanto is building the highest TeraFLOPS per Watt Machine Learning computing system and it will be based on the open RISC-V ISA. November 28, 2017 Esperanto Technologies Inc. 7
ET-Maxion RISC-V Processor ET-Maxion will be the highest single thread performance 64-bit RISC-V processor Allow RISC-V to be positioned alongside highest performance processors. Enable companies to go RISC-V from top to bottom. Reduces threat of retaliation by eliminating need to go to another architecture at high end. Provide a viable high-end alternative for companies wanting to make the transition to RISC-V. Performance goals: Single thread integer performance comparable to the best IP cores available from market leaders. Great Linux performance to run OS and applications. Technical features: 64-bit RISC-V RV64GC instruction set Starting from BOOM v2, but expect substantial changes Out-of-order pipeline Multiple levels of cache Multiprocessor support Optimized for 7nm CMOS Will be used in Esperanto's products and made available as a licensable core. November 28, 2017 Esperanto Technologies Inc. 8
ET-Minion RISC-V Processor ET-Minions do all the heavy floating point work Designed for: High floating point throughput Energy efficiency Future computing is all about reducing energy/operation Technical features: 64-bit RISC-V ISA with Vector Extensions In-order pipeline Integrated vector floating point unit Instruction extensions Tensor instructions and other enhancements for machine learning Support for graphics operations Hardware accelerators Multiple hardware threads of execution Will be used in Esperanto's products and made available as a licensable core. November 28, 2017 Esperanto Technologies Inc. 9
Esperanto s AI Supercomputer on a Chip Esperanto is building a single chip AI computing system in 7nm Building blocks are scalable and allow wide range of implementations, performance and power Goal is to have the best TeraFLOPS/Watt of any machine learning system... all based on RISC-V All plans subject to change this is not a product announcement but here is our POR... Technical features: 16 64-bit ET-Maxion RISC-V cores with private L1 and L2 caches 4096 64-bit ET-Minion RISC-V cores each with their own vector floating point unit Hardware Accelerators Network on chip allows all processors to reside in same address space Multiple levels of cache High bandwidth DRAM interfaces Physical design optimized for 7nm Energy efficient design techniques This chip will go into Esperanto s machine learning computing products. November 28, 2017 Esperanto Technologies Inc. 10
Running Machine Learning on RISC-V Machine Learning applications are dominated by need for TeraFLOPS of computing. Other companies are proposing special purpose hardware for ML using proprietary instruction sets. We believe a better approach is to base all processing on RISC-V. Allows leveraging all the software OS, Compilers, etc. the RISC-V community is developing Where necessary add instruction extensions Where necessary add hardware accelerators Esperanto s approach: Thousands of energy-efficient RISC-V Cores... each with RISC-V Vector capability... and some instruction extensions.. for example Tensor instructions Let s see how it runs. November 28, 2017 Esperanto Technologies Inc. 11
Resnet50 Deep Neural Network RISC-V with Tensor extensions running on Esperanto s ET-Minion Verilog RTL Inference on one batch of eight images, running all layers Activity in 4096 Processors Performance as benchmark runs November 28, 2017 Esperanto Technologies Inc. 12
Graphics What about graphics for RISC-V systems on a chip? For RISC-V to enter the broader hardware ecosystem, there must be an on-die graphics solution. Esperanto is putting thousands of general purpose RISC-V cores on a chip. Might those provide a sufficient graphics solution? after all, high-end graphics units have lots of shader processors that aren t too different from RISC-V Graphics requires a Shader Compiler generating RISC-V instructions so we wrote a Shader Compiler and the code to distribute the workload over thousands of cores Results are more than sufficient for many RISC-V SoC s, see next slide. November 28, 2017 Esperanto Technologies Inc. 13
Rendering Graphics Shader compiler generating RISC-V code running on Esperanto s ET-Minion Verilog RTL Activity in 4096 Processors Activity in 4096 Processors November 28, 2017 Esperanto Technologies Inc. 14
RISC-V with DSE is a better alternative Custom hardware with proprietary instruction sets for Machine Learning seems to be a trend Experience says general purpose solution is better when algorithms are changing rapidly Challenge the claim that a completely new proprietary instruction set is needed for ML Better approach is to use RISC-V as a base ISA and add extensions if needed General purpose cores are more desirable than special purpose cores when minimal performance difference Use general purpose RISC-V ISA with Domain Specific Extensions (DSE), e.g.: RISC-V Vector ISA Tensor Instructions Hardware acceleration Freedom to innovate with the open RISC-V ISA allows us to carefully find the best way to extend RISC-V General purpose CPU + DSE allows growing software base on RISC-V RV64GC Access domain specific extensions through library calls from RV64GC code Let s make general purpose RISC-V processors with Domain Specific Extensions the better alternative. November 28, 2017 Esperanto Technologies Inc. 15
Summarizing: What Esperanto is working on Real Hardware: Building advanced computing solutions for Machine Learning/Graphics/etc., based on RISC-V Expect to sell both chips and systems We think this can be the best system for ML, and a showcase for how good RISC-V can be Licensable IP: ET-Maxion: Highest single-thread performance RISC-V OOO core for integer intensive tasks ET-Minion: Most energy-efficient RISC-V core for floating point intensive tasks ET-Graphics: Scalable graphics solutions for RISC-V Optimized physical design for TSMC 7nm using standard CAD tools and Verilog Free IP: Esperanto will continue to manage and support the open-source BOOM repository Welcome the involvement of the RISC-V community in achieving our goals Email me or info@esperanto.ai if you want to help or ask for more information We re hiring as well, email us at jobs@esperanto.ai, or see the website www.esperanto.ai November 28, 2017 Esperanto Technologies Inc. 16