AP Computer Science Final Project - Indexing and the Word-Location Trick Indexing Algorithms

Transcription

1 AP Computer Science Final Project - Indexing and the Word-Location Trick Indexing Algorithms Search engines can sift through tens of billions of web pages and index those pages so that humans can quickly and accurately find pertinent information with the click of a button. How do search engines such as Google perform this magic you ask? In this program, we will write methods that perform simple, yet effective forms of this powerful indexing algorithm*. In the project template, the WebPageProcessor class has a method filetoletters() that reads plain text from a text file (representing a random web page on the net) and places each letter into an ArrayList, one letter per element. For example, if the text in a file page1.txt is: The cat in the hat then filetoletters( page1.txt ) will store each letter (a String of length 1) as one element in an ArrayList of Strings as shown below: Actual HTML formatted web pages are much more complex than a raw text file, but this simple example will give you the basic idea behind the logic of the indexing algorithm. Part A filetoletters() a. Create a plain text file using Word or other word processor and save it as a plain text file called myfile.txt file in the WebpageIndexingProject project folder. In this simple example, only include one line of text with no enter s or return s that would create a second line of text. Keystrokes such as enter and other non-standard characters will be addressed later. b. Edit the WebpageProcessorDriver so that the argument to the method filetoletters() is myfile.txt. c. Run the WebpageProcessorDriver to verify that your file myfile.txt is converted to an ArrayList of Strings by the program. * an additional algorithm, the Ranking Algorithm, is used by Google to rank pages according to their relevance. This algorithm is what separated Google from the rest of the pack.

2 Part B Class WebpageProcessor Class WebpageProcessor contains method letterstopage(). This method will take as its parameter an ArrayList of single letters (i.e. Strings of length 1 in Part A above) and convert it into a ProcessedPage which consists of an ArrayList of words as shown below. ProcessedPage object shown below consists an of ArrayList<String> Since this ArrayList of words represents the words on a webpage, this method will return a ProcessedPage object that has-an ArrayList of these words. As shown above, the result is a ProcessedPage object that contains an ArrayList<String> of length 5 that has one word per element. Examine the ProcessedPage class in the WebpageIndexingProject template. This is a complete class; you do not have to make any changes to this class for this project. Write the WebpageProcessor method letterstopage() that has the method header as shown below: public ProcessedPage letterstopage(arraylist<string> letters) Use the WebpageProcessorDriver to test Part B. Sample output from running the letterstopage method is shown below:

3 Part C searchword A ProcessedPageCollection object consists of an ArrayList of ProcessedPages as shown below. Note that each ProcessedPage object was created using letterstopage() method written in part B. Each of the pages above contains all of the words on its page. Recall that words on a ProcessedPage object are stored internally as an ArrayList<String>. This allows each word to be accessed uniquely via its index. Write the ProcessedPageCollection method searchword that takes a single String search word and returns the page number(s) that the word is found on. See the following examples: Example 1: The result of calling searchword( the ), will return an ArrayList containing the values 0, 1, and 2, since the String the is found on pages 0, 1, and 2. Note that the values 0 and 1 should only appear once although the is found twice on each page. Example 2: The result of calling searchword( sat ), will return an ArrayList containing the values 0 and 2, since the String sat is found on pages 0 and 2. Write the ProcessedPageCollection method searchword that has the method header as shown below: public ArrayList<Integer> searchword( String word )

4 Part D Processing enter keystrokes (extra credit) The WebpageProcessor method filetoletters() does not read lines of text after an enter or return keystroke. Modify the method filetoletters() so that this method can process files that have an enter or return keystroke. Test your solution using the method written in Part C. If your algorithm works correctly, you will be able to verify that all text in the file has been read in correctly and can successfully be found using the searchword method.

5 Part E createindextable() and printindextable() The Word-Location algorithm allows a search engine to quickly locate a phrase from a selection of indexed web pages without having to look at the web pages themselves **. Consider the following set of ProcessedPage objects in a ProcessedPageCollection below: Give the set of pages shown above, an index table was created based on the page and word location within a page for each word on the page as shown below. Note that column numbers correspond to the page a word is found on and the row numbers correspond to the location of the word on that page. The word dog has the entries 1-1 and 2-5 since it appears on (page 1, word 1) and (page 2, word 5.) Examine the indextable shown below to verify that each word and corresponding page-word location in the table are consistent with the words found on pages 0, 1, and 2 above. ** in the actual implementation of this method, index tables are constantly being updated in the background so that successive searches will yield more accurate and relevant search results depending on what web pages are being created and removed from the web. Imagine that this algorithm is running on the web as you read!

6 This part will require you to complete the following: a. Design and include a private instance variable indextable in the ProcessedPageCollection class that is capable of storing a word, page, and word location for each word on a given page. This is the most creative and critical part of the project. When designing this class, keep in mind its purpose and the fact that you will want easy access to the data in the indextable. Write the ProcessedPageCollection method createindextable that has the method header as shown below: public void createindextable() b. Include a method printindextable() that will print all elements in indextable in a 2D array-like format. Write the ProcessedPageCollection method printindextable() that has the method header as shown below: public void printindextable()

7 Part F searchphrase Using the indextable from part E, write the ProcessedPageCollection method searchphrase() that will take two String arguments representing two search keywords and return the most relevant ProcessedPage in the ProcessedPageCollection. The most relevant page is defined as the page that contains the two search keywords that are the closest to one another on a given page. The method searchphrase() has the method header shown below: public ArrayList<Integer> searchphrase( String keyword1, String keyword2 ) This method should implement an algorithm that analyzes the page and word number pairs in the indextable and returns the page(s) that is/are the most relevant. Example 1: searchphrase( cat, sat ) will return an ArrayList containing the integer 0 since this is the page contains cat and sat in the nearest proximity. The words cat and sat also appear on page 2, but they are further apart than they are on page 0. Example 2: searchphrase( the, cat ) will return an ArrayList containing the integers 0, 1 and 2 since these three pages contain the and cat in equal proximity to one another (i.e. the words the and cat and appear one word apart on all three pages. Example 3: searchphrase( dog, stood ) will return an ArrayList containing the integer 1 since this is the page that dog and stood appear on with the nearest proximity (they are one word apart.) Page 2 also contains dog and stood, but they are in further proximity from one another (they are 5 words apart) in addition to being in reverse order. Part G User interface Make your project user-friendly by creating a single JOptionPane dialog box that will ask the user for a search word or words (like a Google search bar.) Your program should use previously written methods to retrieve and display the most relevant search results (i.e. print the text of the page on the screen.) Part H Search Time Include the timer that was used in the Searching and Sorting projects to display how fast your algorithm finds a page (See Appendix A on the next page for this code.) Display the time along with your search results. finalproject.isdone=true;

8 Appendix A Timer Code public class TimerDriver public static void main( String args[] ) Stopwatch seiko = new Stopwatch(); seiko.starttimer(); for(int ctr = 0; ctr < Integer.MAX_VALUE; ctr++) // place the code/algorithm that you wish to time here } seiko.endtimer(); seiko.printelapsedtime(); } // end method main } // end class TimerDriver public class Stopwatch private long starttime; private long endtime; public Stopwatch() starttime = 0; endtime = 0; } // end constructor StopWatch public void starttimer() starttime = System.nanoTime(); } // end method starttimer public void endtimer() endtime = System.nanoTime(); } // end method endtimer public void printelapsedtime() System.out.println( "Elapsed time in nanoseconds is: " + (endtime - starttime)); } // end method printelaspedtime } // end class Stopwatch