CS 200 Algorithms and Data Structures, Fall 2012 Programming Assignment #4

Transcription

1 Building a Word Histogram using a Hashtable Due Nov. 14 noon Objectives In this assignment, you will implement classes for histogramming words using your own hashtable. You will write: (1) An implementation of the Hashtable: In this your software should be able to manage hash space collisions efficiently. (2) A class to build a histogram based on the frequency of the word occurances in classic literature. This class should use the Hashtable built in (1) Background: Hashtable Hashtable is a class mapping keys to values. Any non-null object can be used as a key. To successfully store and retrieve objects from a hashtable, the objects used as keys must implement the hashcode method and the equals method. An instance of Hashtable has two parameters that affect its performance: initial capacity and load factor. The capacity is the number of buckets in the hash table, and the initial capacity is simply the capacity at the time the hash table is created. The hash table is open: in the case of a "hash collision", a single bucket stores multiple entries, which must be searched sequentially. The load factor is a measure of how full the hash table is allowed to get before its capacity is automatically increased. When the number of entries in the hashtable exceeds the product of the load factor and the current capacity, the capacity is increased by calling the rehash method. Generally, the default load factor (.75) offers a good tradeoff between time and space costs. Higher values decrease the space overhead but increase the time cost to look up an entry (which is reflected in most Hashtable operations, including get and put). The initial capacity controls a tradeoff between wasted space and the need for rehash operations, which are time-consuming. No rehash operations will ever occur if the initial capacity is greater than the maximum number of entries the Hashtable will contain divided by its load factor. However, setting the initial capacity too high can waste space. If many entries are to be made into a Hashtable, creating it with a sufficiently large capacity may allow the entries to be inserted more efficiently than letting it perform automatic rehashing as needed to grow the table. (Fom the Java Hashtable document Task Description Part 1. Build FastHashtable The first task is to build a Hashtable. A skeleton file, FastHashtable is provided. This class contains Java Generics. Please note that this class does not implement java.util.map. 1

2 You can use the hashcode() implemented in each of the Java objects. For example, PA4 uses String object as key. Therefore, you can use the hashcode() method provided by java.lang.string. Your FastHashtable should be able to manage collisions and rehash. You should choose an algorithm(s) to manage this collision. You can use single algorithm or mix multiple algorithms. Of course, you can also come up with your own algorithm. Use of Java s Hashtable class and Hashmap class are NOT ALLOWED for PA4. Please make sure that your software does not include java.util.hashtable or java.util.hashmap in any case. The skeleton files include some code for testing. Part 2. Build a Histogram with your FastHashtable Download free e-books listed in the Appendix and build a histogram of each of the word that appeared in the book. Each of the items (a combination of a word entity and its frequency) should be stored in your FastHashtable. Implement a skeleton file, Histogram.java. This file also includes some code for testing. To extract and manipulate Strings from a novel, 1) Split words using the space character 2) Use only the text part: it includes upper case letters, lower case letters and decimal digit numbers. 3) Ignore all of other symbols such as mathematical notation, quotation marks, or underscore. For example, I m should be interpreted as im 4) Consider all the words as lowercase. ( My and my are considered the same word my ) To satisfy the above requirements, use the provided method gettextonly(string input) of the Histogram class. Also, use the method, tolowercase() of the String class for the requirement (4). Part 3. Tune your hashtable For this assignment, your software will compete with ones from other teams. Five of the fastest submissions will get a bonus score. Besides selecting a collision algorithm, you can tune your hashtable by means of adjusting the initial capacity and the load factor. Your FastHashtable class should provide those methods. You can specify constant values based on the result of your experiments. You can also add software components to adjust the factors automatically. (e.g. based on the input file size). Part 4.Test your software Test your software with the included testing program: PA4_Test.java This test program provides 4 commands, 1) histogram 2) freq 3) mostfreq 4) wordcount 2

3 histogram builds a histogram from the input file and prints the histogram. freq will find the number of appearances of the specified word. mostfreq will find the word(s) that has(have) appeared most frequently in the input file. wordcount will return the number of unique words appeared in the input file. Each of the sub-tests will measure the delay in java PA4_Test histogram test.txt =============================== 0 funnylooking 1 1 more 1 2 is 1 3 are 1 4 tail 1 5 im 1 6 on 2 7 theres 1 8 my 3 9 stegosaurus 1 10 boney 1 11 back 1 12 many 1 13 for 1 14 dinosaur 1 15 a 1 16 plates 1 17 and 1 18 name 1 =============================== Turnaround time for building a hashtable: 2 The test.txt file is included in the file set posted. The output table contains three columns: index, word, and frequency. java PA4_Test mostfreq test.txt my Most frequently appreared word(s) is(are) my Turnaround time for finding the most frequently appeared word : 1 java PA4_Test freq test.txt my my has appeared in test.txt 3 times. java PA4_Test wordcount test.txt There are 19 valid word(s) java PA4_Test mostfreq OliverTwist.txt Turnaround time for finding most frequently appeared word : 10 3

4 java PA4_Test freq OliverTwist.txt the the has appeared in OliverTwist.txt 9771 times. java PA4_Test mostfreq WarAndPeace.txt Turnaround time for finding most frequently appeared word : 14 java PA4_Test freq WarAndPeace.txt the the has appeared in WarAndPeace.txt times. java PA4_Test mostfreq HuckFinn.txt Most frequently appreared word(s) is(are) and Turnaround time for finding most frequently appeared word : 6 java PA4_Test freq HuckFinn.txt and and has appeared in HuckFinn.txt 6299 times. java PA4_Test mostfreq Hamlet.txt Turnaround time for finding most frequently appeared word : 4 java PA4_Test freq Hamlet.txt the the has appeared in Hamlet.txt 1106 times. java PA4_Test wordcount Hamlet.txt There are 5328 valid word(s) java PA4_Test wordcount HuckFinn.txt There are 7723 valid word(s) java PA4_Test wordcount WarAndPeace.txt There are valid word(s) java PA4_Test wordcount OliverTwist.txt There are valid word(s) Also, you can find a sample output of PA4_Test histogram Hamlet.txt in the skeleton fileset. 4

5 Deliverables Submit a tar ball of your java source code including: FastHashtable.java Histogram.java Keep all of your source code in a single flat directory. The skeleton files are provided. Please do not modify PA4_Test.java. Note: You are required to work as a team in this assignment. You and your teammate should submit only ONE copy of the assignment. Please write down the implementer s name(s) on top of each of the source code. Grading This assignment will account for 5 of your final grade. The grading itself will be done on a 50 point scale. After the test, five teams who developed the fastest hashtables will get 10 points as a bonus score. We will test with 5 different sized of novels (listed in the Appendix) and combine the delays for building histogram and retrieval data. Late Policy Please check the late policy available from the course web page Appendix Please download plain-text ebooks listed below. This data is from the project Gutenberg. ( This ebooks are free in the United States because their copyright has expired. [1] Oliver Twist by Charles Dickens (914kB) [2] War and Peach by Leo Tolstoy (3.1 MB) [3] Adventures of Huckleberry Finn by Mark Twain (584 kb) [4] Hamlet by William Shakespeare (180 kb) 5