Sergey Maidanov. Software Engineering Manager for Intel Distribution for Python*

Transcription

1 Sergey Maidanov Software Engineering Manager for Intel Distribution for Python*

2 Introduction Python is among the most popular programming languages Especially for prototyping But very limited use in production Python is #1 programming language in hiring demand followed by Java and C++. And the demand is growing Many computational problems require HPC/Big Data production environments Hire a team of Java/C++ programmers OR Ease access for Python researcher and/or have team of Python programmers to deploy optimized Python in production

3 What is required for making Python performance closer to native code? BLACK SCHOLES FORMULA M O P T I O NS / SEC Bringing parallelism (vectorization, threading, multi-node) to Python is essential to make it useful in production 55x Static compilation 350x Vectorization, threading, and data locality optimizations Python C C (Parallelism) Chapter 19. Performance Optimization of Black Scholes Pricing Configuration info: - Versions: Intel Distribution for Python Technical Preview 1 (Aug 03, 2015), icc 15.0; Hardware: Intel Xeon CPU E GHz (2 sockets, 16 cores each, HT=OFF), 64 GB of RAM, 8 DIMMS of 8GB@2133MHz; Operating System: Ubuntu LTS.

4 Performance-productivity technological choices Numerical packages acceleration with Intel performance libraries (MKL, DAAL, IPP) Better parallelism and composable multi-threading (TBB, MPI) Profiling Python and mixed language codes (VTune) Language extensions for vectorization and multithreading (Cython, Numba, Pyston) Integration with Big Data and Machine Learning platforms/frameworks (Spark, Hadoop, Theano, etc)

5 Energy Signal Processing Financial Analytics Engineering Design Digital Content Creation Science & Research

6 Our approach 1. Enable hooks to Intel MKL, Intel DAAL, Intel IPP functions in the most popular numerical/data processing packages NumPy, SciPy, Scikit-Learn, PyTables, Scikit-Image, Available through Intel Distribution for Python* and as Conda packages 2. Most optimizations eventually upstreamed to home open source projects 3. Provide Python interfaces for DAAL (a.k.a PyDAAL) More cores More Threads Wider vectors

7 Optimized mathematical building blocks Intel Math Kernel Library (Intel MKL) Linear Algebra Fast Fourier Transforms Vector Math BLAS LAPACK Up to 100x ScaLAPACK faster Sparse BLAS Sparse Solvers Iterative PARDISO* SMP & Cluster Multidimensional FFTW interfaces Cluster FFT Up to 10x faster! Trigonometric Hyperbolic Exponential Log Power Root Up to 10x faster! Vector RNGs Multiple BRNG Up to Support methods for 60x independent streams faster! creation Support all key probability distributions Summary Statistics Kurtosis Variation coefficient Order statistics Min/max Variance-covariance And More Splines Interpolation Trust Region Fast Poisson Solver Functional domain in this color accelerate respective NumPy, SciPy, etc. domain 7

8 Speedup SciPy FFT vs Ubuntu* Default Python Speedup NumPy FFT vs Ubuntu* Default Python* Optimized FFT show case Intel Math Kernel Library (Intel MKL) Original SciPy FFT implementation is about 2x faster than original NumPy FFT Intel engineers bridged NumPy and SciPy implementations via common layer and embedded MKL FFT calls, what measurably accelerates both NumPy and SciPy NumPy and SciPy are computationally compatible NumPy FFT NumPy vs Ubuntu* Default Python* FFT descriptors caching applied for enhanced performance in repetitive and multidimensional FFT calculations SciPy FFT Intel SciPy vs Ubuntu* Vanilla Python* PSF Intel (1 thread) Intel (32 threads) 4.8 Available starting Intel Distribution for Python* 2017 Beta 0.0 PSF Intel (1 thread) Intel (32 threads) 8

9 random_sample() uniform(0, 1) standard_normal() normal(1, 5) standard_exponential() exponential(2) standard_gamma(0.78) gamma(0.78, 2) standard_t(1.78) standard_cauchy() chisquare(7.78) beta(1.2, 4.3) lognormal(-2, 1) f(2.5, 3.4) noncentral_f(2.5, 3.5, 12) wald(0.7, 2.7) gumbel(0.7, 2.5) weibull(0.7) logistic(0, 2) laplace(1, 2) noncentral_chisquare(3, 2.3) triangular(0.7, 0.9, 1.2) rayleigh(2.5) binomial(123, 0.4) geometric(0.7) negative_binomial(45, 0.34) poisson(0.5) poisson(7) poisson(53) Optimized RNG show case Intel Math Kernel Library (Intel MKL) Implemented numpy.random in vector fashion to enable vector MKL RNG and VML calls Enabled multiple BRNG Enabled multiple distribution transformation methods numpy.random speedup due to MKL RNG 0 Initial data. Final data to be available in the update for Intel Distribution for Python* 2017 Beta 9

10

11 Intel MPI in Python 11

12

13 Oversubscription with Nested Parallelism Software components are built from smaller components If each component specifies threads... there can be too much! Intel TBB dynamically balances thread loads and effectively manages oversubscription

14 Intel TBB in Python TBB treading already exposed via Intel MKL and Intel DAAL 2017 Beta introduces Intel TBB module for effective higher-level parallelism Implements multiprocessing.pool interface from TBB import Pool Uses Monkey-patching in order to enable Dask/Joblib/etc.. python -m TBB my_app.py with TBB.Monkey(): Switches MKL accelerated packages into TBB-based threading layer when applied Bug fix: Numpy did not release GIL for MKL LAPACK calls

15 Case study: Collaborative filtering in Python Recommendations of useful purchases 15

16 User requests per sec Collaborative Filtering - Generation of User Recommendations Positive effect of TBB-based nested parallelism in Python 27x x 15x 11x Nested parallelism with Intel TBB x Fedora Python Intel Python Fedora Python + ThreadPool Configuration Info: - Versions: Intel(R) Distribution for Python , Beta (Mar 04, 2016), MKL version for Intel Distribution for Python 2017, Beta, Fedora* built Python*: Python (default, Sep ), NumPy 1.9.2, SciPy , multiprocessing 0.70a1 built with gcc 5.1.1; Hardware: 96 CPUs (HT ON), 4 sockets (12 cores/socket), 1 NUMA node, Intel(R) Xeon(R) E5-4657L v2@2.40ghz, RAM 64GB, Operating System: Fedora release 23 (Twenty Three) Innermost parallelism only (via Intel MKL) Outermost parallelism only Oversubscription with nested parallelism Intel Python+ThreadPool Intel Python + ThreadPool + TBB 16

17

18 Legal Disclaimer & INFORMATION IN THIS DOCUMENT IS PROVIDED AS IS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. Intel, Pentium, Xeon, Xeon Phi, Core, VTune, Cilk, and the Intel logo are trademarks of Intel Corporation in the U.S. and other countries. Intel s compilers may or may not optimize to the same degree for non-intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice. Notice revision #

19