61A Lecture 32. Wednesday, November 14

Transcription

1 61A Lecture 32 Wednesday, November 14

2 Processing Sequential Data 2

3 Processing Sequential Data Many data sets can be viewed and processed sequentially: 2

4 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts 2

5 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts Votes cast in a presidential election 2

6 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts Votes cast in a presidential election Sensor readings of an airplane 2

7 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts Votes cast in a presidential election Sensor readings of an airplane The set of all positive integers 2

8 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts Votes cast in a presidential election Sensor readings of an airplane The set of all positive integers However, the sequence interface we developed previously does not always apply. 2

9 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts Votes cast in a presidential election Sensor readings of an airplane The set of all positive integers However, the sequence interface we developed previously does not always apply. A sequence has a finite, known length 2

10 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts Votes cast in a presidential election Sensor readings of an airplane The set of all positive integers However, the sequence interface we developed previously does not always apply. A sequence has a finite, known length A sequence support element selection for any element 2

11 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts Votes cast in a presidential election Sensor readings of an airplane The set of all positive integers However, the sequence interface we developed previously does not always apply. A sequence has a finite, known length A sequence support element selection for any element In most cases, satisfying the sequence interface requires storing the entire sequence in a computer's memory. 2

12 Processing Sequential Data Many data sets can be viewed and processed sequentially: The set of all Twitter posts Votes cast in a presidential election Sensor readings of an airplane The set of all positive integers However, the sequence interface we developed previously does not always apply. A sequence has a finite, known length A sequence support element selection for any element In most cases, satisfying the sequence interface requires storing the entire sequence in a computer's memory. Today: Efficient representations of sequential data 2

13 Implicit Sequences 3

14 Implicit Sequences An implicit sequence is a representation of sequential data that does not explicitly store each element. 3

15 Implicit Sequences An implicit sequence is a representation of sequential data that does not explicitly store each element. Example: The range class represents consecutive integers. 3

16 Implicit Sequences An implicit sequence is a representation of sequential data that does not explicitly store each element. Example: The range class represents consecutive integers. The range is represented by two values: start and end. 3

17 Implicit Sequences An implicit sequence is a representation of sequential data that does not explicitly store each element. Example: The range class represents consecutive integers. The range is represented by two values: start and end. The length and elements are computed on demand. 3

18 Implicit Sequences An implicit sequence is a representation of sequential data that does not explicitly store each element. Example: The range class represents consecutive integers. The range is represented by two values: start and end. The length and elements are computed on demand. Constant space for arbitrarily large sequences. 3

19 Implicit Sequences An implicit sequence is a representation of sequential data that does not explicitly store each element. Example: The range class represents consecutive integers. The range is represented by two values: start and end. The length and elements are computed on demand. Constant space for arbitrarily large sequences. Demo 3

20 The Iterator Interface 4

21 The Iterator Interface An iterator is an object that can provide the next element of a (possibly implicit) sequence. 4

22 The Iterator Interface An iterator is an object that can provide the next element of a (possibly implicit) sequence. The iterator interface has two methods: 4

23 The Iterator Interface An iterator is an object that can provide the next element of a (possibly implicit) sequence. The iterator interface has two methods: next (self) returns the next element in the sequence 4

24 The Iterator Interface An iterator is an object that can provide the next element of a (possibly implicit) sequence. The iterator interface has two methods: next (self) returns the next element in the sequence iter (self) returns an equivalent iterator (Why?) 4

25 The Iterator Interface An iterator is an object that can provide the next element of a (possibly implicit) sequence. The iterator interface has two methods: next (self) returns the next element in the sequence iter (self) returns an equivalent iterator (Why?) The next function invokes the next method on its argument. 4

26 The Iterator Interface An iterator is an object that can provide the next element of a (possibly implicit) sequence. The iterator interface has two methods: next (self) returns the next element in the sequence iter (self) returns an equivalent iterator (Why?) The next function invokes the next method on its argument. If there is no next element, then the next method of an iterator should raise a StopIteration exception. 4

27 The Iterator Interface An iterator is an object that can provide the next element of a (possibly implicit) sequence. The iterator interface has two methods: next (self) returns the next element in the sequence iter (self) returns an equivalent iterator (Why?) The next function invokes the next method on its argument. If there is no next element, then the next method of an iterator should raise a StopIteration exception. Demo 4

28 The For Statement 5

29 The For Statement for <name> in <expression>: <suite> 5

30 The For Statement for <name> in <expression>: <suite> 1. Evaluate the header <expression>, which yields an iterable object. 5

31 The For Statement for <name> in <expression>: <suite> 1. Evaluate the header <expression>, which yields an iterable object. 2. For each element in that sequence, in order: 5

32 The For Statement for <name> in <expression>: <suite> 1. Evaluate the header <expression>, which yields an iterable object. 2. For each element in that sequence, in order: A. Bind <name> to that element in the first frame of the current environment. 5

33 The For Statement for <name> in <expression>: <suite> 1. Evaluate the header <expression>, which yields an iterable object. 2. For each element in that sequence, in order: A. Bind <name> to that element in the first frame of the current environment. B. Execute the <suite>. 5

34 The For Statement for <name> in <expression>: <suite> 1. Evaluate the header <expression>, which yields an iterable object. 2. For each element in that sequence, in order: A. Bind <name> to that element in the first frame of the current environment. B. Execute the <suite>. An iterable object has a method iter that returns an iterator. 5

35 The For Statement for <name> in <expression>: <suite> 1. Evaluate the header <expression>, which yields an iterable object. 2. For each element in that sequence, in order: A. Bind <name> to that element in the first frame of the current environment. B. Execute the <suite>. An iterable object has a method iter that returns an iterator. >>> counts = [1, 2, 3] >>> for item in counts: print(item)

36 The For Statement for <name> in <expression>: <suite> 1. Evaluate the header <expression>, which yields an iterable object. 2. For each element in that sequence, in order: A. Bind <name> to that element in the first frame of the current environment. B. Execute the <suite>. An iterable object has a method iter that returns an iterator. >>> counts = [1, 2, 3] >>> for item in counts: print(item) >>> counts = [1, 2, 3] >>> items = counts. iter () >>> try: while True: item = items. next () print(item) except StopIteration: pass

37 Generators and Generator Functions 6

38 Generators and Generator Functions A generator is an iterator backed by a generator function. 6

39 Generators and Generator Functions A generator is an iterator backed by a generator function. A generator function is a function that yields values. 6

40 Generators and Generator Functions A generator is an iterator backed by a generator function. A generator function is a function that yields values. When a generator function is called, it returns a generator. 6

41 Generators and Generator Functions A generator is an iterator backed by a generator function. A generator function is a function that yields values. When a generator function is called, it returns a generator. >>> def letters_generator(): current = 'a' while current <= 'd': yield current current = chr(ord(current)+1) 6

42 Generators and Generator Functions A generator is an iterator backed by a generator function. A generator function is a function that yields values. When a generator function is called, it returns a generator. >>> def letters_generator(): current = 'a' while current <= 'd': yield current current = chr(ord(current)+1) >>> for letter in letters_generator(): print(letter) 6

43 Generators and Generator Functions A generator is an iterator backed by a generator function. A generator function is a function that yields values. When a generator function is called, it returns a generator. >>> def letters_generator(): current = 'a' while current <= 'd': yield current current = chr(ord(current)+1) >>> for letter in letters_generator(): print(letter) a b c d 6

44 Streams 7

45 Streams A stream is a recursive list with an explicit first element and an implicit rest of the list. 7

46 Streams A stream is a recursive list with an explicit first element and an implicit rest of the list. class Stream(object): """A lazily computed recursive list.""" 7

47 Streams A stream is a recursive list with an explicit first element and an implicit rest of the list. class Stream(object): """A lazily computed recursive list.""" class empty(object): def repr (self): return 'Stream.empty' empty = empty() 7

48 Streams A stream is a recursive list with an explicit first element and an implicit rest of the list. class Stream(object): """A lazily computed recursive list.""" class empty(object): def repr (self): return 'Stream.empty' empty = empty() def init (self, first, compute_rest=lambda: empty): assert callable(compute_rest), 'compute_rest must be callable.' self.first = first self._compute_rest = compute_rest self._rest = None 7

49 Streams A stream is a recursive list with an explicit first element and an implicit rest of the list. class Stream(object): """A lazily computed recursive list.""" class empty(object): def repr (self): return 'Stream.empty' empty = empty() def init (self, first, compute_rest=lambda: empty): assert callable(compute_rest), 'compute_rest must be callable.' self.first = first self._compute_rest = compute_rest self._rest = def rest(self): """Return the rest of the stream, computing it if necessary.""" if self._compute_rest is not None: self._rest = self._compute_rest() self._compute_rest = None return self._rest 7

50 Integer Streams 8

51 Integer Streams An integer stream is a stream of consecutive integers. An integer stream starting at k consists of k and a function that returns the integer stream starting at k+1. 8

52 Integer Streams An integer stream is a stream of consecutive integers. An integer stream starting at k consists of k and a function that returns the integer stream starting at k+1. def make_integer_stream(first=1): """Return a stream of consecutive integers, starting with first. >>> s = make_integer_stream(3) >>> s.first 3 >>> s.rest.first 4 """" def compute_rest(): return make_integer_stream(first+1) return Stream(first, compute_rest) 8

53 Mapping a Function over a Stream 9

54 Mapping a Function over a Stream Mapping a function over a stream applies a function only to the first element at first, but computes the rest lazily. 9

55 Mapping a Function over a Stream Mapping a function over a stream applies a function only to the first element at first, but computes the rest lazily. def map_stream(fn, s): """Map a function fn over the elements of a stream s.""" if s is Stream.empty: return s def compute_rest(): return map_stream(fn, s.rest) return Stream(fn(s.first), compute_rest) 9

56 Mapping a Function over a Stream Mapping a function over a stream applies a function only to the first element at first, but computes the rest lazily. def map_stream(fn, s): """Map a function fn over the elements of a stream s.""" if s is Stream.empty: return s def compute_rest(): return map_stream(fn, s.rest) return Stream(fn(s.first), compute_rest) Demo 9

57 Filtering a Stream 10

58 Filtering a Stream When filtering a stream, processing continues until an element is kept in the output. 10

59 Filtering a Stream When filtering a stream, processing continues until an element is kept in the output. def filter_stream(fn, s): """Filter stream s with predicate function fn.""" if s is Stream.empty: return s def compute_rest(): return filter_stream(fn, s.rest) if fn(s.first): return Stream(s.first, compute_rest) else: return compute_rest() 10

60 A Stream of Primes 11

61 A Stream of Primes The stream of integers not divisible by any k <= n is: 11

62 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, 11

63 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. 11

64 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. Called the Sieve of Eratosthenes. 11

65 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. Called the Sieve of Eratosthenes. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 11

66 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. Called the Sieve of Eratosthenes. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 11

67 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. Called the Sieve of Eratosthenes. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 11

68 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. Called the Sieve of Eratosthenes. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 11

69 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. Called the Sieve of Eratosthenes. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 11

70 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. Called the Sieve of Eratosthenes. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 11

71 A Stream of Primes The stream of integers not divisible by any k <= n is: The stream of integers not divisible by any k < n, Filtered to remove any element divisible by n. Called the Sieve of Eratosthenes. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13 Demo 11