HIRP OPEN 2018 Compiler & Programming Language. An Efficient Framework for Optimizing Tensors

Transcription

1 An Efficient Framework for Optimizing Tensors 1 Theme: 2 Subject: Compiler Technology List of Abbreviations NA 3 Background Tensor computation arises frequently in machine learning, graph analytics and high performance computing domains. Optimizing tensor is challenging as it requires a thorough understanding of both tensor application and hardware platforms. Furthermore, hand-tuned optimization for a specific application and platform is not a good solution as the algorithms and hardware platforms are constantly evolving. we hope a framework to automatically optimize the performance of tensor application. 4 Scope 4.1 Loop Optimization Technology Global loop optimizations. Given a complex tensor application, it normally involves with a series of matrix and tensor computation. Each tensor computation is usually wrapped using a loop. It is highly likely that these loops are designed with different loop structures such as loop bounds, dependencies, etc. For the loops carrying no dependency can be parallelized or vectorized, individually. However, in practice, it is highly likely that these loops are dependent (e.g. producer and consumer). In order to enable a global optimization across different loops, transformations may be required to expose the parallelism and pipelining opportunities that may not be available in the original form. 1 / 4

2 4.2 Memory Access Optimization Technology Memory optimization. Our target platforms are GPUs and another machine model with scratchpad memory. On both platforms, optimizing the scratchpad memory (shared memory on GPU is scratchpad memory) has a great impact on the performance as accessing on-chip memory is much faster than accessing off-chip global memory. Obviously, the size of scratchpad memory is limited that we cannot allocate all the variables in it. 4.3 Auto-tuning Technology Auto-tuning. Different applications may use different size of matrix, which may lead to different loop transformations and memory optimizations. Therefore, we will design an auto-tuning approach that can automatically explore the design space and find the optimal solution. 5 Expected Outcome and Deliverables Publications: 1-2 research publication submissions are expected to document the novelty and results of our research Software components: An End-to-End Tensors Optimization compiler and Network Description for GPU or Hardware Accelerator. Final report: A final report will be delivered to highlight the main accomplishments of the project. 6 Acceptance Criteria The deliveries will be checked against the items described in this document. 7 Phased Project Plan Milestone 1. Survey of CuDNN library. CuDNN uses a set of algorithms and also auto-tuning technique. 2 / 4

3 The outcome of this task is a survey of CuDNN. Milestone 2. Scratchpad memory optimization for convolution. The outcome of this task is the tiled algorithm of convolution. Milestone 3. Performance model for GPU platforms. Data reuse exploited by the shared memory is a vital factor for the final performance. However, we should also model the other factors that impact the performance such as thread level parallelism, occupancy, register allocation, and computation and memory overlap, etc. The outcome of this task is the performance model for general NVIDIA embedded platforms. Milestone 4. Demonstration of other algorithms. In order to generalize the optimization, performance model proposed above, we plan to test them using other tensor computations such as LSTM. The output of this task is the optimization solution and experiment results of LSTM on GPUs. 3 / 4

4 Copyright(C)Huawei Technologies Co., Ltd All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademarks and Permissions and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders. Confidentiality All information in this document (including, but not limited to interface protocols, parameters, flowchart and formula) is the confidential information of Huawei Technologies Co., Ltd and its affiliates. Any and all recipient shall keep this document in confidence with the same degree of care as used for its own confidential information and shall not publish or disclose wholly or in part to any other party without Huawei Technologies Co., Ltd s prior written consent. Notice Unless otherwise agreed by Huawei Technologies Co., Ltd, all the information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute the warranty of any kind, express or implied. Distribution Without the written consent of Huawei Technologies Co., Ltd, this document cannot be distributed except for the purpose of Huawei Innovation R&D Projects and within those who have participated in Huawei Innovation R&D Projects. 4 / 4

5 Micro-services Design Guided Model Checking Tools 1 Theme: 2 Subject: Cloud Tools Technology List of Abbreviations NA 3 Background Micro-services development has many challenges for developers due to its distributed nature. Tools for designing, debugging, testing and monitoring are the keys to overcome these obstacles. Early detection of correctness, security and performance issues of micro-services application are critical as usual, but more difficult compared to traditional application development. The difficulties lie in testing the interactions among individual micro services as design and implementation as complexities increase. There are many technologies in industry and Huawei. Most of these technologies are by tracing and system logging in later development stages such as testing and production. These traditional software development tools can still very helpful, but many come too late in development process and can potentially cost huge amount of resources debugging and fixing the issues. There have been many researches in academia and in industry on tools that can help micro services developers to detect issues in early development stages. One school of thoughts are on modeling the micro services systems with declarative and logic languages and checking for the errors. Maude is one of these declarative and logic languages designed for distributed systems like micro services.. 1 / 4

6 4 Scope 4.1 Programming models and error checking tools for micro services built on the programming model of selection Declarative and logic languages like Maude has been well tested and established in academia for distributed systems. We want to explore and to apply its model in micro services, starting with micro services application design. 4.2 Automatically generate micro service model with programming modeling language Once succeed in 4.1, we will need to research ways to automatically generate the model using the language. In even longer term, we also want to look into building more complicated model checking for performance with its temporal model checking capabilities. 5 Expected Outcome and Deliverables Publications: 1~3 research publication submissions are expected to document the novelty and results of our research Software components: new cloud micro services development tools based this technologies in Huawei public cloud, assisting micro service developers to build and to migrate micro service applications on Huawei cloud. Final report: A final report will be delivered to highlight the main accomplishments of the project. 6 Acceptance Criteria The deliveries will be checked against the items described in this document. 7 Phased Project Plan Milestone 1. Studying and exploring capabilities of model checking languages like Maude and their 2 / 4

7 applications in distributed systems. Identifying one benchmark micro services we can model using these languages and find potential development issues such as correctness of the program based on its initial design describing all of its micro service properties and connecting mechanism. Milestone 2. Extending the model checking into iterative development of micro services for other capabilities such as temporal model checking for performance and security checking. And show that adding runtime information of the application can also enhance the ability for the tools to find more correctness and performance issues of the application. Milestone 3. Research automatic model generation using the languages, based on micro services design (and/or runtime) information. If the automation is successful, this automation will result in a tool in Huawei s public cloud or user s own private cloud. Milestone 4. Verify the product worthiness of this tools with Huawei public cloud. This tool is most suitable for ServiceStage where we can potentially build the development environment from the design stage. Explore ways to package this tool in Huawei s SDK. 3 / 4

8 Copyright(C)Huawei Technologies Co., Ltd All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademarks and Permissions and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders. Confidentiality All information in this document (including, but not limited to interface protocols, parameters, flowchart and formula) is the confidential information of Huawei Technologies Co., Ltd and its affiliates. Any and all recipient shall keep this document in confidence with the same degree of care as used for its own confidential information and shall not publish or disclose wholly or in part to any other party without Huawei Technologies Co., Ltd s prior written consent. Notice Unless otherwise agreed by Huawei Technologies Co., Ltd, all the information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute the warranty of any kind, express or implied. Distribution Without the written consent of Huawei Technologies Co., Ltd, this document cannot be distributed except for the purpose of Huawei Innovation R&D Projects and within those who have participated in Huawei Innovation R&D Projects. 4 / 4

9 Runtime Analysis Assisted μs Debugging Tools 1 Theme: Computing Technology 2 Subject: Cloud Tools Technology List of Abbreviations μs Micro Service 3 Background Micro-services development has shown great potential to have been widely adapted by recent studies and surveys done with participation from industries. The studies also found that tools for identifying faults in the runtime and other stages of development are critical for its wider adaptation. Runtime monitoring, verification and debugging of micro-services application are few of the main challenges by the same survey of those participants with role of managers and architect. Among these challenges, debugging remains concerns as a major issue for them to move forward in adopting μs architecture and development by the same survey. Developers often spend days or even weeks on analyzing and debugging of the runtime errors. In addition, diagnosis and identifying and debugging errors at third party micro service has also been potential issues many applications in cloud, and extending to IoT areas. Recent progress in runtime monitoring and identifying errors in μs applications is very promising and provides a clear direction of combing runtime diagnosis, analysis with information on design and development in assisting debugging errors in μs applications. we want to explore how runtime diagnostics, analysis and tracing of errors provide feedback to applications design and implementation and help identify design and 1 / 4

10 implementation errors that lead to runtime faults. On the other hand, we also see the possibilities of the reverse feedback where design and implementation can also play a critical role in debugging runtime errors. 4 Scope 4.1 Identify and establish connection between micro services runtime error and issues in other stages In theory, the connection between some runtime errors and design time problems exists. For example, the design of external logic of a micro service with another micro service (internal or external source) can directly cause it runtime fault because the message passing between two micro services becomes asynchronous and delivery of these messages lacks sequence control, resulting fault behavior in the cloud at runtime. In this phase, we want to explore more such cases, such as dependency and reliability of the dependencies among micro services. 4.2 An effective micro service debugging tool for all stages of micro service development with runtime analysis assistance In longer team, we also want to provide developers with insightful and actionable diagnostics and suggestions for fixing the issue in their design and implementation in a tool that provide design and implementation tracing and visualization. 5 Expected Outcome and Deliverables Publications: 1-2 research publication submissions are expected to document the novelty and results of our research Software components: new micro service error debugging tool Final report: A final report will be delivered to highlight the main accomplishments of the project. 2 / 4

11 6 Acceptance Criteria HIRP OPEN 2018 The deliveries will be checked against the items described in this document. 7 Phased Project Plan Milestone 1. Surveying Huawei public cloud, other industry and open source, as well as academic for existing micro service runtime fault detections tools. Discovering the pain points for Huawei cloud micro service developers and identifying the most impactful starting point. Milestone 2. Extending the analysis to all stages of the micro service development, including Huawei s cloud DevOps, especially that of design and implementation stages. Finding how each stage of development can provide debugging information for the other for developer to quickly identify and fix the errors Milestone 3. Debugging tool for all stages of micro services development. This tool can be in the form of SDK that can be plugged in to other development environment, including Huawei s own public cloud services. 3 / 4

12 Copyright(C)Huawei Technologies Co., Ltd All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means without prior written consent of Huawei Technologies Co., Ltd. Trademarks and Permissions and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd. All other trademarks and trade names mentioned in this document are the property of their respective holders. Confidentiality All information in this document (including, but not limited to interface protocols, parameters, flowchart and formula) is the confidential information of Huawei Technologies Co., Ltd and its affiliates. Any and all recipient shall keep this document in confidence with the same degree of care as used for its own confidential information and shall not publish or disclose wholly or in part to any other party without Huawei Technologies Co., Ltd s prior written consent. Notice Unless otherwise agreed by Huawei Technologies Co., Ltd, all the information in this document is subject to change without notice. Every effort has been made in the preparation of this document to ensure accuracy of the contents, but all statements, information, and recommendations in this document do not constitute the warranty of any kind, express or implied. Distribution Without the written consent of Huawei Technologies Co., Ltd, this document cannot be distributed except for the purpose of Huawei Innovation R&D Projects and within those who have participated in Huawei Innovation R&D Projects. 4 / 4