mith College Computer Science Sets and Hashing CSC212 Fall 2014 Dominique Thiébaut

Transcription

1 mith College Computer Science Sets and Hashing CSC22 Fall 204 Dominique Thiébaut

2 The Problem: Large amount of live tweets Want the list of all tweeters Each only listed once

3 What data structure? Assumption: all tweeters have a unique Id number (integer)

4 Option : BST

5

6 Complexity?

7 O()?

8 Yes! Hashing!

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26 Have we seen 00 yet?

27 Have we seen 00 yet?

28 Have we seen 00 yet?

29 Have we seen 00 yet?

30 YES!

31 YES! O()!

32 How? What Properties?

33 How: a hash function int[] table = new int[ DIM ]; int index = hash( key ); table[ index ] = key;

34 Hash Properties Fast Avoid clustering Avoid collisions Must work with all types of objects

35 Fast arithmetic operations, integers Avoid clustering Avoid collisions Must work with all types of objects

36 keys Fast Avoid clustering uniformly distributed indexes Avoid collisions Must work with all types of objects

37 keys hash index [ ] hashing Fast Avoid clustering uniformly distributed indexes Avoid collisions Must work with all types of objects

38 Fast Avoid clustering Avoid collisions Don t want two different keys with same hash index Must work with all types of objects

39 Fast Avoid collisions Fortunately, everything in memory is a number! Avoid clustering Must work with all types of objects

40 Fast Avoid collisions Fortunately, everything in memory is a number! Avoid clustering Must work with all types of objects Alpha = A l p h a = = 486

41 Implementation

42 Division/Modulo index = key % Dim Works best is Dim is prime Example: 202%0 > 2 hash 0, 90, 40, 70 change Dim to 0%=0, 90%=2 40%=, 70%= If Dim is not prime, then take modulo by a prime first: index = (key % p) % Dim Dim = 0

43 Division/Modulo index = key % Dim Works best is Dim is prime Example: 202%0 > 2 49%0 > 9 hash 0, 90, 40, 70 change Dim to 0%=0, 90%=2 40%=, 70%= If Dim is not prime, then take modulo by a prime first: index = (key % p) % Dim Dim = 0

44 Division/Modulo index = key % Dim Works best is Dim is prime Example: 202%0 > 2 49%0 > 9 hash 0, 90, 40, 70 69%0 > change Dim to 0%=0, 90%=2 40%=, 70%= If Dim is not prime, then take modulo by a prime first: index = (key % p) % Dim Dim = 0

45 Division/Modulo index = key % Dim Works best if Dim is prime Example: hash 0, 90, 40, 70 when Dim=0 change Dim to 0%=0, 90%=2 40%=, 70%= If Dim is not prime, then take modulo by a prime first: index = (key % p) % Dim Dim = 0

46 Folding-Based Hashing Shift-Folding, Boundary Folding Shift Folding key = > > = 68 % Dim = index Boundary Folding key = > > = 66 % Dim = index

47 Extraction Hashing Extract some digits from the key key = > > 289 % Dim = index

48

49 Avoiding Collisions

50 Open 0 Addressing Dim = 0

51 Open 0 Addressing 4 2 key = 44 hash( 44 ) > Dim = 0

52 Open 0 Addressing 4 2 key = 44 hash( 44 ) > Dim = 0

53 Open 0 Addressing 4 2 key = 04 hash( 04 ) > Dim = 0

54 Open 0 Addressing 4 2 key = 04 hash( 04 ) > Dim = 0

55 Open Addressing key = 04 hash( 04 ) > Dim = 0

56 Open Addressing key = hash( ) > Dim = 0

57 Open Addressing key = hash( ) > Dim = 0

58 Open Addressing key = hash( ) > Dim = 0

59 Open Addressing key = hash( ) > Dim = 0

60 Open Addressing key = hash( ) > Dim = 0 linear probing

61 Open Addressing key = hash( ) > What if we want to delete keys? Dim = 0

62 Avoiding Collisions 2

63 Chaining 0 2 key = 4 hash(key)= Dim = 0

64 Chaining 0 2 key = 4 hash(key)= Dim = 0

65 Chaining 0 2 key = 004 hash(key)= Dim = 0

66 Chaining key = 004 hash(key)= Dim = 0

67 Chaining key = 99 hash(key)= Dim = 0

68 Chaining key = 99 hash(key)= Dim = 0

69 Complexity

70 Open Addressing: It depends on the LF = 6/9 = 66.6% Load Factor # probes Successful Search (LF=0%) Linear Probing /2(+/(-LF)) (.) Unsuccessful Search (LV=0%) /2(+/(-LF) 2 ) (2.)

71 Chaining: It depends on the Average List Length N items Table of dimension Dim Average list length = N/Dim Complexity = O( N/Dim )

72 O()

73 Hashing in Java

74 Hashing in Java Hashing integers Hashing floats Hashing strings Hashing custom objects

75 public class HashingExamples { public static void main(string[] args) { String s = "Smith College"; System.out.println( "s.hashcode() = " + s.hashcode() ); int n = 24; System.out.println( "n's hashcode() = " + Integer.valueOf( n ).hashcode() ); double y = 6.02E2; System.out.println( "y's hashcode() = " + Double.valueOf( y ).hashcode() ); } } float x =.49f; System.out.println( "x's hashcode() = " + Float.valueOf( x ).hashcode() ); s.hashcode() = n's hashcode() = 24 y's hashcode() = x's hashcode() =

76 Java Sets: HashSet

77 HashSets HashSet = collection of unique objects* *Remember the example of unique tweeters

78 HashSet<Integer> set = new HashSet<Integer>( ); int count = 0; for ( int i=0; i<000000; i++ ) { count++; set.add( i ); // replicate some elements if ( i%00 == ) { count ++; set.add( i ); } } System.out.println( count + " ints added to set. Set contains " + set.size() + " unique items" ); ints added to set. Set contains unique items

79 HashSet<Integer> set = new HashSet<Integer>( ); for ( int i=0; i<0; i++ ) set.add( i*0 ); System.out.print( "set = " ); for ( int x: set ) System.out.print( x + " " ); System.out.println(); set =

80 HashSet<Object> set = new HashSet<Object>(); HashSet<Object> myset = new HashSet<Object>(); myset.add( "Hello" ); myset.add( ); set.add(.49 ); set.add( 204 ); set.add( "Smith College" ); set.add( myset ); System.out.println("Set: " + set ); Use with care! Set: [[, Hello], Smith College, 204,.49]

81 Default Hash Function

82 class MyPair { public int x; public int y; MyPair( int xx, int yy ) { x = xx; y = yy; } public String tostring() { return "(" + x + ", " + y + ")"; } } private void example() { HashSet<MyPair> set = new HashSet<MyPair>(); set.add( new MyPair(, 2 ) ); set.add( new MyPair(, 0 ) ); set.add( new MyPair(, 2 ) ); System.out.println( "Set: " + set ); } Set: [(, 2), (, 0), (, 2)]

83 class MyPair { public int x; public int y; MyPair( int xx, int yy ) { x = xx; y = yy; } public String tostring() { return "(" + x + ", " + y + ")"; } } private void example() { HashSet<MyPair> set = new HashSet<MyPair>(); set.add( new MyPair(, 2 ) ); set.add( new MyPair(, 0 ) ); set.add( new MyPair(, 2 ) ); System.out.println( "Set: " + set ); } It s not working! Set: [(, 2), (, 0), (, 2)]

84 class MyPair { public int x; public int y; MyPair( int xx, int yy ) { x = xx; y = yy; } public String tostring() { return "(" + x + ", " + y + ")"; } public int hashcode(){ return (x+97) * (y+0) ; } public boolean equals( Object o ) { if ( o==null ) return false; MyPair p = (MyPair) o; if ( p.x!= x p.y!= y ) return false; return true; } }

85 class MyPair { public int x; public int y; MyPair( int xx, int yy ) { x = xx; y = yy; } public String tostring() { return "(" + x + ", " + y + ")"; } public int hashcode(){ return (x+97) * (y+0) ; } public boolean equals( Object o ) { if ( o==null ) return false; MyPair p = (MyPair) o; if ( p.x!= x p.y!= y ) return false; return true; } } Set: [(, 2), (, 0)]

86 Always implement equals() if you implement hashcode()!

87 Java Trick Instead of computing the hash code integer every time hashcode() is called, the hash code can be pre-computed once, when the object is created (or modified), and stored in a special field In this case hashcode() is definitely O()

88 HashMaps

89 HashMaps Associate a value to a key. A HashMap is a set of (key:value) pairs where the hashing is done on the key. Chaining is used to resolve collision

90 Colleges 0 Smith Amherst 2 Hampshire MtHolyoke Towns 0 Northampton Amherst 2 Amherst South Hadley

91 Colleges 0 Smith Amherst 2 Hampshire MtHolyoke Towns 0 Northampton Amherst 2 Amherst South Hadley Smith Northampton Amherst Hampshire MtHolyoke Keys CollegeTowns Amherst Amherst South Hadley Values MAP

92 import java.util.hashmap; HashMap<String, String> coltown = new HashMap<String, String>(); coltown.put( "Amherst", "Amhest" ); coltown.put( "Smith", "Northampton" ); coltown.put( "Hampshire", "Amherst" ); coltown.put( "MtHoyoke", "South Hadley" ); for ( String college: coltown.keyset() ) System.out.println( college + " College is in " + coltown.get( college ) ); Smith College is in Northampton MtHoyoke College is in South Hadley Hampshire College is in Amherst Amherst College is in Amhest

93 Can use user-defined key objects, but must implement equals() and hashcode()!

94 import java.util.hashmap; HashMap< MyPair, String > map = new HashMap<MyPair, String>(); map.put( new MyPair(, 2), "George" ); map.put( new MyPair( 0, 20 ), "Smith College" ); map.put( new MyPair(, ), null ); System.out.println( "map = " + map ); MyPair x = new MyPair( 0, 20 ); if ( map.containskey( x ) ) System.out.println( x + " --> " + map.get( x ) ); map = {(, )=null, (, 2)=George, (0, 20)=Smith College} (0, 20) --> Smith College

95 HashMaps/HashSets in Final Project? Where could a HashMap or a HashSet be useful in the final project? What would the key be? What would the value be?