How fast can we multiply and divide polynomials?

Size: px

Start display at page:

Download "How fast can we multiply and divide polynomials?"

Kristina Phelps
5 years ago
Views:

1 How fast can we multiply and divide polynomials? Cybernet Systems, December 14, Michael Monagan Center for Experimental and Constructive Mathematics, Simon Fraser University, Vancouver, British Columbia, CANADA. Joint work with Roman Pearce.

2 The Mathematics Of Computer Algebra and Analysis project. > factor(2 x 3 + x 2 y 2 x 3 y + x 2 5 yx 3 y y 2 x 3 y 1); > solve({x 2 + y 2 + z 2 4, x y z + 2, x y + z 3 1}); t 1 2 t 1 > Determinant( t 2 t 1 ); 1 + t t + t 2 > x 2 ln(x)e x + (1 x) ln(x)e x 2xe x dx;

3 The Mathematics Of Computer Algebra and Analysis project. > factor(2 x 3 + x 2 y 2 x 3 y + x 2 5 yx 3 y y 2 x 3 y 1); > solve({x 2 + y 2 + z 2 4, x y z + 2, x y + z 3 1}); t 1 2 t 1 > Determinant( t 2 t 1 ); 1 + t t + t 2 > x 2 ln(x)e x + (1 x) ln(x)e x 2xe x dx; Risch e x θ 1 ln x θ 2 a polynomial {}}{ x 2 θ 2 θ 1 + (1 x)θ 2 θ 1 2xθ 1 dx where θ 1 = θ 1 θ 2 = 1/x.

4 Polynomials are the key! Talk Outline: How do CAS represent polynomials? How do CAS multiply and divide polynomials? Our representation and algorithms. How fast we compared with other CAS? Immediate Monomial Project (for Maple 15) Parallel Multiplication (for Maple 15)

5 How do CAS represent polynomials?

6 Recursive and distributed polynomial representations. The distributed represention: monomials x i y j z k are sorted in lexicographical order (Magma, Mathematica): f = 6x 3 + 9xy 3 z 8xy 2 z + 7y 2 z or graded lex order (Singular, Maple 15): f = 9xy 3 z 8xy 2 z + 7y 2 z 2 6x Key property: if X, Y, Z are monomials then Y > Z = XY > XZ.

7 Recursive and distributed polynomial representations. The distributed represention: monomials x i y j z k are sorted in lexicographical order (Magma, Mathematica): f = 6x 3 + 9xy 3 z 8xy 2 z + 7y 2 z or graded lex order (Singular, Maple 15): f = 9xy 3 z 8xy 2 z + 7y 2 z 2 6x Key property: if X, Y, Z are monomials then Y > Z = XY > XZ. The recursive representation (Macsyma, REDUCE, Derive, Pari): f = ( 6)x 3 + ((9z)y 3 + ( 8z)y 2 )x 1 + ((7z 2 )y 2 + 5y 0 )x 0.

8 Maple s sum of products representation. PROD 7 x 1 y 3 z 1 PROD 5 y 3 z 2 PROD 7 x 1 y 2 z 1 PROD 3 x 3 SUM xy 3 z 4y 3 z 2 6xy 2 z 8x 3 5 Singular s distributed representation. POLY x y z

9 Trip s recursive sparse representation. POLY x POLY y POLY y 2 3 POLY z 2 4 POLY z 1 6 POLY z 1 9 ( 5y 4z 2 y 3 ) + ( 6zy 2 + 9zy 3 )x 8x 3 Pari s recursive dense representation POLY 6 x 0 8 POLY 6 y POLY 6 y POLY 5 z POLY 4 z 0 6 POLY 4 z 0 9

10 Our representation uses packed monomials. Packing for x i y j z k in graded lex order with x > y > z: Why? One 64 bit word : i + j + k i j k. }{{} (i + j + k) i j + k.

11 Our representation uses packed monomials. Packing for x i y j z k in graded lex order with x > y > z: One 64 bit word : i + j + k i j k. }{{} (i + j + k) i j + k. Why? Because monomial > and are one machine instruction.

12 Our representation uses packed monomials. Packing for x i y j z k in graded lex order with x > y > z: One 64 bit word : i + j + k i j k. }{{} (i + j + k) i j + k. Why? Because monomial > and are one machine instruction. Our packed array for 9xy 3 z 4y 3 z 2 6xy 2 z 8x 3 5. POLY 5 x y z packing d = total degree dxyz dxyz dxyz dxyz dxyz Why graded lex order?

13 Our representation uses packed monomials. Packing for x i y j z k in graded lex order with x > y > z: One 64 bit word : i + j + k i j k. }{{} (i + j + k) i j + k. Why? Because monomial > and are one machine instruction. Our packed array for 9xy 3 z 4y 3 z 2 6xy 2 z 8x 3 5. POLY 5 x y z packing d = total degree dxyz dxyz dxyz dxyz dxyz Why graded lex order? No exponent overflow in division.

14 How do CAS multiply and divide polynomials?

15 Let f = f 1 + f f n and g = g 1 + g g m where f 1 > f 2 > > f n and g 1 > g 2 > > g m. Using h = f g = ((f 1 g + f 2 g) + f 3 g) + + f n g and h g = f : (((h f 1 g) f 2 g) f 3 g) f n g

16 Let f = f 1 + f f n and g = g 1 + g g m where f 1 > f 2 > > f n and g 1 > g 2 > > g m. Using h = f g = ((f 1 g + f 2 g) + f 3 g) + + f n g and h g = f : (((h f 1 g) f 2 g) f 3 g) f n g takes O(n 2 m) comparisons of monomials and O(nm) multiplications of coefficient and monomials. Example: f = x n + x n x and g = y n + y n y.

17 Our algorithms for multiplication and division use heaps.

18 Heaps A binary heap H with n entries is a partially ordered array satisfying H i H 2i and H i H 2i+1. H 1 H 2 H 3 H 4 H 5 H 6 H 7 H H 1 is the biggest entry in a heap. We can extract the maximum entry in O(log 2 n) comparisons. We can insert a new entry in O(log 2 n) comparisons.

19 Multiplication using a binary heap. Johnson, 1974, a simultaneous n-ary merge: f = a 1 X 1 + a 2 X a n X n g = b 1 Y 1 + b 2 Y b m Y m (sorted) add f 2 g 1 f 1 g f 1 g2 Heap f 1 x f 2 x f 3 x fn x ( g 1 + g 2 + g g m) ( g 1 + g 2 + g g m) ( g 1 + g 2 + g g m). ( g 1 + g 2 + g g m ) O(nm log n) comparisons. Space for n monomials in the heap. Can pick n m.

20 Division using a heap. Johnson s quotient heap algorithm. Dividing f g = q compute f #q q i g i=1 O(#f + #q#g log #q) comparisons O(#q) working memory Our divisor heap algorithm. Dividing f g = q compute f #g g i q i=2 O(#f + #q#glog #g) comparisons O(#g) working memory

21 Minimal heap division (Monagan & Pearce, 2008) Problem: we don t know if #q > #g when starting a division. E.g. (x 7 y 7 ) (x y) = x 6 + yx 5 + y 2 x y 6. Start with quotient heap, switch to divisor heap when #q = #g. f min(#q,#g) i=1 q i g }{{} quotient heap #g g i (q #g+1 + ) i=2 } {{ } divisor heap O(#f + #q#glog min(#q, #g)) comparisons O(min(#q, #g)) working memory

22 Which CAS is fastest?

23 Benchmark 1: A dense Fateman problem. f = (1 + x + y + z + t) 20 g = f + 1 f and g have 39 bit coefficients and 10,626 terms h = f g has 83 bit coefficients and 135,751 terms Intel Core2 3.0 GHz multiply divide p = f g q = p/f Maple s s Maple s s Singular s s Magma V s s Pari (w/ GMP) s s Trip v s - sdmp (unpacked) 5.15 s 5.44 s sdmp (packed) 2.26 s 2.77 s Maple s 4.46s

24 Benchmark 2: A sparse 10 variable problem. f = (x 1 x 2 + x 2 x 3 + x 3 x 4 + x 4 x 5 + x 5 x 6 + x 6 x 7 + x 7 x 8 + x 8 x 9 + x 9 x 10 + x 1 x 10 + x 1 + x 2 + x 3 + x 4 + x 5 + x 6 + x 7 + x 8 + x 9 + x ) 4 g = (x x x x x x x x x x x 1 + x 2 + x 3 + x 4 + x 5 + x 6 + x 7 + x 8 + x 9 + x ) 4 6, 746 8, 361 = multiply p = f g divide q = p/f 3, 157, 883 terms seconds secs Maple s s Maple s s Singular s 18.00s Magma V s s Pari (w/ GMP) 7.06s 7.05s Trip v0.99 (rationals) 8.13s sdmp (unpacked) 11.12s 10.37s sdmp (packed) 2.46s 2.61s Maple s 14.45s

25 Benchmark 3: Factorization speedup in Maple 14. In Maple 13, > h := expand(f*g); > divide(h,f, q ); call expand/bigprod (f,g) and expand/bigdiv (h,f,q) for large inputs. In Maple 14, we reprogrammed expand/bigprod and expand/bigdiv to convert to SDMP, multiply (divide) in SDMP, then convert back to Maple. Benchmark factor( h ) where h = (a + 1)(a + 2) a = Maple13 Magma Maple14 Speedup (x + y + z) x (1 + x + y + z) x (1 + x + y + z) x (1 + x + y + z + t) x Table: Factorization Benchmark Timings (in CPU seconds)

26 The Immediate Monomial Project. A joint MITACS project with Maplesoft. A new data structure being implemented by Paul de Marco. POLY SEQ 4 X Y Z How will we pack monomials? E.g. x i y j z k on a 64 bit computer. Always try to pack all monomials into one word. If i + j + k < 2 16 pack i + j + k i j k in one word. If i + j + k 2 16 use Maple s existing representation. So the number of variables determines the packing.

27 The Immediate Monomial Project POLY SEQ 4 X Y Z Let f (x 1, x 2,, x n ) = f 1 + f f m. O(nm) O(1) : lcoeff(f); degree(f); indets(f); O(nm) O(n) : f; has(f,x); type(f,polynom(integer)); O(nm) O(m) : degree(f,x); diff(f,x); coeffs(f,x); A 10 20% gain in overall efficiency gain for Maple 14?

28 Parrallelizing Multiplication Using Heaps Intel Core i7.

29 Parallel Algorithm One heap per core, merge results in a global heap. Global Heap Local Heaps f g 4 Don t waste real or cpu time: partition terms transfer data balance load

30 Transferring Data Threads write to a finite circular buffer. 0 N 1 r w r mod N w mod N #define N /* size in words (256 K) */ #define CLINE 64 /* bytes per cache line */ struct buffer { long r; /* words read */ char pad1[cline - sizeof(long)]; long w; /* words wrote */ char pad2[cline - sizeof(long)]; long data[n]; }; 32K L1 32K L1 32K L1 32K L1 256K L2 256K L2 256K L2 256K L2 8MB Shared L3 Intel Core i7

31 Load Balancing Global Heap Local Heaps f g 4 threads try to acquire a lock for the global heap one thread per core avoids context switches and OS threads independently adjust their share of global work buffer full do more global work buffer empty do less global work

32 Dense Benchmark f = (1 + x + y + z + t) 30 g = f = terms W (f, g) = 3332 threads Core i7 2.66GHz Core 2 2.4GHz sdmp s 6.15x s 4.25x (packed) s 4.24x s 3.10x s 2.50x s 2.13x s s s 3.89x s 3.94x Trip 1.0 beta s 2.97x s 2.97x recursive dense s 1.98x s 1.99x s s s 3.26x s 3.38x Trip 1.0 beta s 2.66x s 2.71x recursive sparse s 1.88x s 1.91x s s Magma s Pari/GP s s Singular s s Maple s s

33 Parallel Speedup: Core i7 6 W(f,g) = 2737 : terms parallel speedup W(f,g) = 2040 : terms W(f,g) = : terms W(f,g) = : 1.63 M terms W(f,g) = : 3.09 M terms W(f,g) = : 6.01 M terms W(f,g) = : 11.4 M terms W(f,g) = : 18.4 M terms W(f,g) = : 32.7 M terms W(f,g) = : 49.3 M terms W(f,g) = : 65.7 M terms dense sparse random univariate polynomials = products linear terms 5x terms threads 3 4

Polynomial Division using Dynamic Arrays, Heaps, and Packed Exponent Vectors

Polynomial Division using Dynamic Arrays, Heaps, and Packed Exponent Vectors Michael Monagan and Roman Pearce Department of Mathematics, Simon Fraser University Burnaby, B.C. V5A 1S6, CANADA. mmonagan@cecm.sfu.ca