External Memory. Philip Bille

Transcription

1 External Memory Philip Bille

2 Outline Computationals models Modern computers (word) RAM I/O Cache-oblivious Shortest path in implicit grid graphs RAM algorithm I/O algorithms Cache-oblivious algorithm

3 Computational Models

4 Modern Computers CPU L 1 L 2 R A M Disk Ex.: ipad 2 A5 Chip (dual-core ARM A9) L1 cache: 32 KB L2 cache: 1 MB Memory: 512 MB Disk: 16 GB SSD

5 Word RAM Model CPU R A M CPU operations: +,-,/,=,<,>,&,,... Memory access: Read/write a cell in RAM Cost: Number of CPU operations and memory accesses

6 I/O Model [Aggarwal and Vitter 1988] I/O CPU R A M Disk Idea: Focus on read/writes between RAM and disk. Called I/Os. An I/O operation read/writes a block of B consecutive cells between RAM and disk. Ignore computation and other levels of memory hierarchy.

7 I/O Model [Aggarwal and Vitter 1988] I/O CPU R A M Disk N = problem size M = RAM size B = I/O block size I/O operation: Read/write a block of B consecutive cells between RAM and disk. Cost: Number of I/O operations.

8 Cache-Oblivious Model [Frigo et al. 1999] I/O CPU R A M Disk As I/O model, but algorithms do not know M and B (you cannot hardwire M or B in your algorithm!). Program in RAM model and analyze in I/O model. Assume optimal cache replacement strategy with full associativity.

9 Cache-Oblivious Model [Frigo et al. 1999] I/O CPU R A M Disk Lemma: Standard cache replacement strategies (e.g. LRU only FIFO) only uses a constant factor additional I/Os compared to optimal strategy. Lemma: We can simulate full associativity with 1-way associativity using only a constant factor additional I/Os.

10 Cache-Oblivious Model [Frigo et al. 1999] I/O CPU R A M Disk Properties: Efficient between 2 levels => efficient at all levels. Portable, self-tuning, and simple

11 Shortest Path in Implicit Grid Graphs

12 Implicit Grid Graphs a n a n a s b a n a n a Let S and T be two strings of length n. The implicit grid graph for S and T consists of A 2D grid of (n+1) x (n+1) nodes. For each node an edge to neighbors to E, S, SE. E and S edges have weight 1. SE edge (i-1,j-1) to (i, j) has weight 0 if S[i] = T[j] and 1 otherwise.

13 Shortest Paths in Implicit Grid Graphs a n a n a s b a n a n a The shortest path in Implicit Grid Graphs (SPIIGG) problem: Input: Strings S and T of length n. Output: Tthe length of the shortest path in the implicit grid graph for S and T from (0,0) to (n, n).

14 Applications Shortest paths in implicit grid graphs is the edit distance problem. With other edge weight functions we get longest common subsequence, sequence alignment, string similarity, approximate string matching, etc.

15 RAM Solution b a n a n a a n a n a s b a n a n a a n a n a s How can we solve SPIIGG? Construct (n+1) x (n+1) matrix. Fill in each entry in O(1) time in left-to-right top-to-bottom order. Time: O(n 2 ) Space: O(n) (only store current + last row) Slightly faster solutions known [MP1980, Myers1999, CLZ2002, BFC2008]

16 External Memory Algorithms for SPIIGG

17 Outline I/O Porting the RAM algorithm Table partitioning Cache-Oblivious Recursive table partitioning

18 Porting the RAM Algorithm Idea: What happens if we do the same as the RAM algorithm? Read and write blocks between external and internal memory as needed.

19 ? Strings are stored in O(n/B) blocks. Store table in O(n/B) blocks per row. Read and write blocks as needed. How many I/Os? O(n 2 /B)

20 ? Theorem: We can solve SPIIGG in O(n 2 /B) I/Os O(n 2 ) time O(n) space

21 Table Partitioning Idea: When we read a block into internal memory, use it as much as possible.

22 Divide table into subtables with overlapping boundaries. Process all subtables from left-to-right, top-to-bottom order. Processing a subtable: Read corresponding substrings and input boundary into internal memory Fill in subtable using RAM algorithm. Write output boundary to disk.

23 Given only substrings and input boundary we can compute output boundary. Choose size of subtable to be dm x dm for d < 1 such that substrings + input boundary + output boundary + space for internal memory algorithm on subtable < M. How many I/Os?

24 How many I/Os to process a subtable of size dm x dm? O(M/B) Total number of I/Os? O(n 2 /M 2 M/B) = O(n 2 /MB)

25 Theorem: We can solve SPIIGG in O(n 2 /MB + n/b) I/Os O(n 2 ) time O(n) space

26 Cache-Oblivious Algorithm Idea: How can we implement a version of the I/O algorithm without knowledge of M and B? Do a recursive (divide and conquer) version of the I/O algorithm.

27 Divide table into 4 quadrants with overlapping boundaries. Process quadrants from left-to-right, top-to-bottom order. Processing a quadrant: Read corresponding substrings and input boundary. Fill in quadrant recursively. Write output boundary.

28 How many I/Os? Define IO(n) = number of I/Os to process a table of size n x n Case 1: n dm (substrings + boundaries + computation fit in internal mem) IO(n) = O(n/B) Case 2: n > dm?

29 Divide table into 4 quadrants with overlapping boundaries. O(1) Process quadrants from left-to-right, top-to-bottom order. Processing a quadrant: Read corresponding substrings and input boundary. Fill in quadrant recursively. Write output boundary. O(n/B) 4 IO(n/2) O(n/B)

30 IO(n) = ( O(n/B) if n apple dm 4 IO(n/B)+ O(n/B) if n>dm cn B 4 cn 2B =2cn B 16 cn 4B =4cn B Work at level i: Size of subproblem at level i: Number of levels (before subproblem fits in mem): Total: log(cn/dm) X i=0 2 i cn B 2 i cn B = cn B cn 2 i cn dm cn = dm ) i = log(cn/dm) 2i apple cn B 2cn n 2 dm = O MB

31 Cache-Oblivious Algorithm Theorem: We can solve SPIIGG in O(n 2 /MB + n/b) I/Os O(n 2 ) time O(n) space

32 Summary Computationals models Modern computers (word) RAM I/O Cache-oblivious Shortest path in implicit grid graphs RAM algorithm I/O algorithms Cache-oblivious algorithm

33 References A. Aggarwal and J. Vitter, The Input/Output Complexity of Sorting and Related Problems, CACM 31 (9), Erik Demaine, Cache-Oblivious Algorithms and Data Structures, Lecture Notes from the EEF Summer School on Massive Data Sets. Rezaul Alam Chowdhury and Vijaya Ramachandran, Cache-oblivious dynamic programming, SODA 2006.