Lab 5 - MOOS Programming - Calculating Prime Factors

Transcription

1 Lab 5 - MOOS Programming - Calculating Prime Factors Unmanned Marine Vehicle Autonomy, Sensing and Communications Feb 22nd, 2018 Michael Benjamin, mikerb@mit.edu Henrik Schmidt, henrik@mit.edu Department of Mechanical Engineering MIT, Cambridge MA Overview Getting Started The Specs of Your pprimefactor Application Key Aspects of this Lab Further C++ Notes for this Lab Unsigned Long Integers Why Our MOOS Input is Generated as a String Value String to Number, and Number to String Conversion Super Handy Utilities for String Parsing Suggestions for Development Progression of this Lab Create the New App, Augment Your Build Tree Augment Your App to Accept and Process Numerical Entries Augment Your Testing to use utimerscript Handling your Work Inside the Iterate Method Write a Stand-Alone C++ Test Program for Calculating Primes Calculate Primes Instead of Even/Odd Modify Your App to be Non-Blocking Technical Suggestions Calculating the Set of Prime Factors for a Given Number A Class to Represent a Prime Number Solution-in-Progress How to Test Your Application Test with a Simple Poke Test with a Simple One-Event Script Test with a Script Generating Random Input Grading: Test with Script Containing Large Primes

2 2

3 1 Overview In this lab we will build our next MOOS application. The function performed by the app is a bit harder: it handles a series of integers and determines, for each, the list of prime factors. An example input: (via incoming mail) NUM_VALUE = "90090" An example output: (via posted mail) PRIME_RESULT = "orig=90090,primes=2:3:3:5:7:11:13" A primary issue addressed in this exercise is how to build an app that performs an operation that may be considerably longer than the time allotted to a typical Iterate() loop. We ask you to write and test your app to run at 4Hz. Test input will be generated by another app at 4Hz or higher. But some test numbers will contain very large prime factors and may not be answerable in the quarter second time window of the Iterate() loop. The goal is to construct your app to continue to handle the smaller numbers at the 4Hz rate, while also handling the larger numbers, incrementally working toward their solution on each loop. 1.1 Getting Started Begin by repeating the same steps from the previous example: 1. Enter the moos-ivp-extend/src directory and run the GenMOOSApp script to build the bare-bones application: $ GenMOOSApp PrimeFactor p "John Doe" 2. Use the app name pprimefactor. It makes testing much easier for the TAs. 3. Add your new app to the build system as before by editing the CMakeLists.txt and test that it builds. At the end of the previous lab, we recommended a few additional steps: Initially augment your brand new pprimefactor app to read in mail, NUM VALUE, and determine if it is even or odd, and post new mail accordingly. Testing was done via upokeddb. Use utimerscript to test the above by with a bunch of values from the script instead of upokedb. Handle the even/odd determination inside the Iterate() loop. Build a C++ list of incoming numbers in OnNewMail() and handle this list inside Iterate(). If you haven t done this pre-lab work, it s still a good idea to go back and do this. But technically it s not required if you want just jump into the task of prime factorization below. 3

4 1.2 The Specs of Your pprimefactor Application Your application should meet the following specifications: 1. Your app should be called pprimefactor. 2. Your app should register for and handle the variable NUM VALUE. Your app should accept mail on this variable containing a 64-bit integer value. The maximum value will be , or 18,446,744,073,709,551,615, the maximum for a 64 bit unsigned integer. You can assume only positive integers as input. 3. The incoming variable is of type string. You must convert it to a numerical value in your code. 4. Your app should determine the list of prime factors for a given input, and generate the following output for each: the original number, an index indicating the order in which it was received, an index indicating the order in which it was calculated, the time it took to solve the factorization. This is in normal clock time, not CPU time. Use the MOOSTime() function to get the time when it was received and the time it was finished. the list of prime factors. a unique id specifying you, e.g., your Athena username. 5. Your output variable should be PRIME RESULT with the below format. Note if a number can be evenly divided by a prime factor more than once, like the example below with two 3 s, list each in the result: PRIME_RESULT = "orig=90090,received=34,calculated=33,solve_time=2.03, primes=2:3:3:5:7:11:13,username=jane" 6. Your application should not be blocked by large numbers. We should be able to see a response to low-number queries almost immediately, regardless of the size of the preceding query. 7. Do not use a cache of known prime numbers. 8. Implement the --help, --interface, and --example command line switches discussed in the lecture. 1.3 Key Aspects of this Lab The key issue in this exercise is to design your pprimefactor app to handle any size number, but not block on difficult numbers. Certain small numbers, and numbers with a short list of small primes, lend themselves to very quick calculation of their sets of prime factors. Other numbers, such as any 20 digit prime number, require many more steps before determining their result. Your app will be rapidly receiving incoming mail holding numbers both hard and easy. The objective is to design your app to quickly handle and post the results for the easy numbers, and not delay the results for easy numbers when a harder number is received. You will need to 4

5 establish a work queue (in the form of a C++ list) to hold newly arrived and unsolved numbers. You will need to populate this queue in your OnNewMail() function. And you will need to work on this queue inside your Iterate() method. The key idea is that each time Iterate() is invoked, a fixed amount of prime factorization calculations will be applied to each item in the queue. For some elements, this will be enough to complete the calculation. They can be removed from the queue, and have their answer posted in the form of outgoing mail to hte MOOSDB. For elements that cannot be finished in the current round, they will remain on the queue to be coninued in the next call of Iterate(). The trick will be to save the state for any unfinished work. In the end we will check for a properly working pprimefactor app by using our a utimerscript sequence of numbers, with easy and hard numbers mixed. We ll check for correct calculation of the factors, and check that the hard numbers didn t block the easy numbers. 5

6 2 Further C++ Notes for this Lab 2.1 Unsigned Long Integers Your pprimefactor application needs to accept and handle non-negative integers up to the maximum value for a 64 bit integer, up to Traditionally this is accomplished by using the following C++ data type: unsigned long int value = ; However, in some older C++ compilers, the "unsigned long int" data type was only 32 bits. In most newer compilers they are 64 bits. To be sure, the following data type has become more common, and in newer compilers the older style will produce a warning. We will use the newer style in this lab: #include <cstdint> uint64_t value = ; Note that you need to include the cstdint library. 2.2 Why Our MOOS Input is Generated as a String Value MOOS messages are either of type string or double. The string type allows virtually any other complex data type to be represented, so essentially the possible types of messages are limitless. The double type is convenient but has limitations in precision discussed here. The double type in C++ is 8 bytes, or 64 bits long. The type uint64 t is also 64 bits. They are both capable of representing 2 64 or 18,446,744,073,709,551,615 distinct numbers. The difference is that with int64 t variables, there is an understanding that only the integer numbers in the range [0, ] will be represented. With double variables, all the negatives numbers, all the floating point non-integer values, e.g , and numbers higher than 18,446,744,073,709,551,615 are stuffed into the same 64 bits. This inevitably means that some numbers are not representable, and instead are mapped to a nearby close number. This is the case with some large integer values beginning with numbers greater than about 15 digits. As an experiment, you can try the sample code in a simple main.cpp file. Note both variables are assigned the same literal value, but the double variable is rounded to a nearby number. int64_t value1 = ; double value2 = ; cout << "value 1: " << value1 << endl; cout << "value 2: " << fixed << value2 << endl; value 1: value 2: ; 6

7 The implications of this mean that if we want to precisely pass large numbers through MOOS messages, we should use the string representation of the number instead. So in our case if we want to pose a large digit prime number we need to use strings. 2.3 String to Number, and Number to String Conversion In this lab you will need to perform some conversions between strings and numerical values. To represent a 64 bit integer you will want to use the type uint64 t. #include <cstdint> uint64_t value = 0; In this lab, if you use the utimerscript script provided in the lab, the numbers will be posted to the variable NUM VALUE as a string value. For example if the posting were: NUM VALUE=" ". To convert from the string, str=" " to a int64 t, use: #include <cstdlib> // for the strtoul() function string str = " " uint64_t value = strtoul(str.c_str(), NULL, 0); In the other direction, converting numerical values into strings, you can use the C++ stringstream class for this: // Include the stringstream header file #include <sstream>... // Create the biggest unsigned long int possibe uint64_t ival = ; // Convert it to a string stringstream ss; ss << ival; string str = ss.str(); // Confirm it works cout << "Value:" << str << endl; 2.4 Super Handy Utilities for String Parsing In this lab you will be generating posted results in the form of a string with comma-separated variable=value pairs, such as the following. PRIME_RESULT = "orig=90090,received=34,calculated=33,solve_time=2.03, primes=2:3:3:5:7:11:13,username=jane" 7

8 At many various points in later labs, you will also want to parse these kinds of strings into their various components. You may even want to do so in this lab if you choose to write a testing mechanism to check your own answers. Fortunately there are a few tools in the MOOS-IvP library mbutil for making this an easy task. These tools are listed below with links to a more full discussion of each. vector<string> parsestring(string, char); string bitestring(string, char); string stripblankends(string); They can be found here: You will also find useful a few utilities for quickly converting numerical or Boolean values to strings: string inttostring(string); string uinttostring(string); string ulinttostring(string); string doubletostring(double, int precision); string booltostring(bool); They can be found here: 8

9 3 Suggestions for Development Progression of this Lab An effective strategy for building complex code is the identification of relatively easier distinct stages where each stage is (a) verifiable, (b) closer to the overall solution, and (c) doesn t require a bunch of re-working between each stage. With that in mind, we offer the following distinct stages progressing on this lab: 3.1 Create the New App, Augment Your Build Tree The first steps in the Prime Factors lab will be the same as this lab. Generate a new app template using GenMOOSApp, as described in the previous lab for podometry. Be sure to call this new app pprimefactor. Add your pprimefactor app to the build tree, as described in the previous lab for podometry. Verify that it builds and is in your shell path. 3.2 Augment Your App to Accept and Process Numerical Entries A good first step is to begin to edit your pprimefactor MOOS app to accept numerical input. Augment your OnNewMail() function to handle this input. As a first step, you can simply determine if the number is odd or even, and immediately post the result. For now you can do this all in OnNewMail(), leaving your Iterate() method empty. The result should be posted as a string, with the original number, and the odd/even evaluation. For example: NUM RESULT = "37,odd" To test this, launch your new app in a simple mission (primes.moos) file, of your own making. This will just launch a MOOSDB and your pprimefactor app. In two seperate terminal windows, launch a uxms scope, and use the second window for poking the MOOSDB $ uxms myfile.moos NUM_VALUE NUM_RESULT Then in the other window: $ upokedb myfile.moos NUM_VALUE:=37 $ upokedb myfile.moos NUM_VALUE:=22 Verify it s all working in your scope window. Note in the above that we are posting NUM VALUE as a string with the := operator. We do this here simply because we will need to do things this way when handling large primes for the reasons discussed in Section Augment Your Testing to use utimerscript In this next step, use utimerscript to test your pprimefactor app rather than manually poking the MOOSDB with upokedb. You will need to add a utimerscript configuration block to your primes.moos mission file. 9

10 The utimerscript app was introduced in Lab 3, and the full documentation is online. The goal here is to repeatedly post the same few numbers. By setting it to repeat, it can just be active upon the mission launch. The following script or similar should suffice. ProcessConfig = utimerscript { event = var=num_value, val=53, time = 0.50 event = var=num_value, val=234, time = 1.00 event = var=num_value, val=117, time = 1.50 reset_max = nolimit reset_time = all-posted } As before, verify with uxms open in another terminal to verify that your app is working properly on these auto-generated values. 3.4 Handling your Work Inside the Iterate Method The next step in developing your pprimefactor app is to handle your mail by creating a C++ list of numbers. In your OnNewMail() function, don t do any work determining the even or odd nature of your incoming numbers. And don t post mail from within your OnNewMail() method. In the mail loop, just build a C++ list of received mail. You may need to read up a bit on C++ STL lists, but see the refresher on lists at the end of the previous lab. An STL list is a doubly linked list, meaning you can access from the beginning, or the end of the list. For now, in your mail function, push new numbers onto the beginning of the list. Since this list will need to be accessed by OnNewMail() and your Iterate() loop, it should be a member variable of your PrimeFactor class definition. Declare it in your app s header file. In your Iterate() loop, access the list by repeatedly popping a value off the end of the list, determine if it is even or odd, and post as before. At the end of each Iterate() loop, the list should be empty. 3.5 Write a Stand-Alone C++ Test Program for Calculating Primes Step aside from MOOS issues for a moment and focus on the issue of calculating prime factors. For this step you can create and build the program along the style of the mini C++ labs, compiling with g++ on the command line. Or you can generate a throw-away MOOS app in the moos-ivp-extend structure that just has a main.cpp file. Further hints for calculating prime factors is provided below in Section 4.1. In this step, your test program can just take a single command line argument, with simple output: $ mytest primes=2:3:3:5:7:11: Calculate Primes Instead of Even/Odd With the results of the previous two steps, you should now be comfortable with both: 10

11 Having an app that reads numerical input (as strings) and populates a work queue (an STL list), which is then acted on inside the app s Iterate() loop. Calculating the prime factors of a given number. In this next step, rather than calculate whether a number is even or odd, calculate the prime factors instead. In your Iterate() loop, step through the list of input values, and determine the primes and post a string answer according to the specs given in Section 1.2. At this point, you re mostly done, but still missing a very important design spec of this app. By stepping through the list of numbers, and calculating the prime factors completely, before going on to the next item in the list, large primes will block easier numbers. The remaining stage of the lab, addressing this issue, is described next. 3.7 Modify Your App to be Non-Blocking In the final stage, as before, work is added to a list in OnNewMail(), and processed from the list in Iterate(). The key difference is that rather than working to completion on each element of the list, a finite amound of work is done on each element. Work, in this case, amounts to a fixed number of modulo arithmetic operations. (By the way 100,000 is a good number to start with for a block of modulo arithmetic operations. But experiment yourself.) During each visit to an element in the list, if all the primes are found within this fixed number of steps, it is removed from the list, and its answer is poted in outgoing mail. Otherwise it is left on the list until the next time the Iterate() method is invoked. But presumably new numbers may be added in the meanwhile as the OnNewMail() function is once again invoked. The key addition of this stage of development is that the elements of a list need some way of saving partial work. Rather than maintiaing a list of 64-bit integers, it will be convenient to maintain a list of something else. That something else needs to be something that can not only store the original integer, but also the list of primes found so far, and some indication where the previous work left off. A class for representing the above is discussed in Section

12 4 Technical Suggestions 4.1 Calculating the Set of Prime Factors for a Given Number You can find several descriptions on the web of the algorithm for calculating the prime factors of a given number. We describe it in our own words here, partly because we want to emphasize the notion of partially calculating the answer. Initially this isn t our focus, but later on it will be important - we will want a algorithm that we can pause and resume where we left off. First, let s discuss the notion of the problem state. The state consists of two parts: A list L of prime factors, initially empty, A number N with a perhaps presently unknown set of prime factors. As the algorithm progresses, the state will evolve with the list L growing and the number N shrinking until L holds the list of prime factors, and N is 1. For example: L = {}, N=2310 <- Initial condition with input 2310 L = {2}, N=1155 L = {2,3}, N=385 L = {2,3,5}, N=77 L = {2,3,5,7}, N=11 L = {2,3,5,7,11}, N=1 <- Final condition, solution {2,3,5,7,11} How do you find the next prime factor at each stage? One simple way to do this is to iterate between i = 2 and the square root of N and checking if i divides evenly into N. Checking if a number divides evenly into another number, in C++, is done with the modulo operator % (a % b == 0) // if b divides evenly into a Confirm for yourself why going beyond the square root is redundant. You may notice that the simple for-loop has some inefficiencies in it. For example, checking if (N%4)==0 is not necessary if we already know (N%2)!=0. In fact that for-loop should only include prime numbers i=2,3,5,7,11,13... rather than i=2,3,4,5,6,7... Here it may become tempting to keep a look-up table of small primes to make your for-loop faster. Resist this temptation. This isn t what the lab is about, and it s not the huge performance boost that it would seem at the outset. This key aspect here for our lab is the idea of iterating step-by-step through a for-loop. Think of a step in the above algorithm as a single a%b calculation. Rather than proceeding from 2 to the square root of N without pause, the algorithm could instead be paused along the way, stopping at some number k between 2 and the square root of N. When the calculation resumes, the for-loop simply picks up where we left off. Normally we wouldn t be interested in calculating things in this manner, but this approach allows us to pause our work on big potentially-hard-to-solve numbers, to quickly handle an easy number that has arrived on the scene while working on that big number. 12

13 Recursion This algorithm naturally lends itself to a recursive solution. A recursive algorithm is just one that calls itself as part of the algorithm, with a condition for termination. A well known one 0, 1, 1, 2, 3, 5, 8, 13, 21, 34,... Fib(n) = Fib(n-1) + Fib(n-2) Fib(1) = 0 Fib(2) = 1 Primes(n) = k + Primes(n/k), if k devides evenly into n Primes(n) = n, if n is prime 4.2 A Class to Represent a Prime Number Solution-in-Progress It will be helpful to build a C++ class to store instances of partially calculated entries. One example of this class is given here. The class implementation (the.cpp file) is up to you. NOTE: When/if you do create a PrimeEntry.cpp file, don t forget to add it to your build list in your pprimefactor/cmakelists.txt file so it will be compiled!! Key parts of this class are: A holder of the original prime number (m orig), A holder of prime factors found so far (m factors), The index at which it was received (m received index), The index at which it was solved (m calculated index), The index in the for-loop to start searching for prime (m start index), A function for finding the next prime, given a maximum number of steps to search (factor()), A function for generating a string report of the results (getreport()), 13

14 // File: PrimeEntry.h #include <string> #include <vector> #include <cstdint> #ifndef PRIME_ENTRY_HEADER #define PRIME_ENTRY_HEADER class PrimeEntry { public: PrimeEntry(); ~PrimeEntry() {}; void setoriginalval(unsigned long int v); void setreceivedindex(unsigned int v) {m_received_index=v;}; void setcalculatedindex(unsigned int v) {m_calculated_index=v;}; void setdone(bool v) {m_done=v;}; bool bool done() {return(m_done);}; factor(unsigned long int max_steps); std::string protected: uint64_t uint64_t bool unsigned int unsigned int getreport(); m_start_index; m_orig; m_done; m_received_index; m_calculated_index; std::vector<uint64_t> m_factors; }; #endif Your MOOS app may contain a list of such elements. It will grow as new mail comes in. And it will shrink during the Iterate() loop as work is done on each element. The Iterate() loop should try to perform some number of steps on each element, then scan the list for those marked as done. For any done element, remove it from the list, and generate a posting to the MOOSDB of the results. 14

15 5 How to Test Your Application 5.1 Test with a Simple Poke To test your application you will need to launch pprimefactor, along with a MOOSDB, and generate an input. So your first step will be to create a simple primes.moos file with the two apps in the Antler block. Then you can simply poke the DB with: $ upokeddb primes.moos NUM_VALUE:=90090 Note the := operator in NUM VALUE:= This will ensure that the value is posted as a string and not a double. The next step is to verify your result with the simple scope: $ uxms primes.moos PRIME_RESULT 5.2 Test with a Simple One-Event Script You can make your testing a bit more streamlined by replacing the single poke in the previous section: $ upokeddb primes.moos NUM_VALUE:=90090 You can replace this with a one-line script in utimerscript. You will add utimerscript to your Antler block in your primes.moos file. And the below script is added also in your primes.moos mission file: The next step is to verify your result with the simple scope: ProcessConfig = utimerscript { event = var=num_value, val="90090" event = quit } The last step, same as before, is to verify your result with the simple scope: $ uxms primes.moos PRIME_RESULT You now have one less step to do, one less command to launch, to test your work. And now that you have a mission file with a script in it, you re ready for more fancy testing described next. 15

16 5.3 Test with a Script Generating Random Input When you re pretty sure that your app is generating correct answers, you will turn your attention to how fast your app is working. In particular, can it handle big numbers quickly? Of course the magnitude of the number is not really the issue. For example 2**48 = is a big number, but should be very fast to solve, essentially requiring no more than 48 modulo operations. In the below recommended test, a utimerscript script is provided to repeatedly generate random numbers of specified length. Following the steps in the previous section, you will create a primes.moos mission file with MOOSDB, pprimefactor, and utimerscript in your Antler block, and then use the below recommended script. ProcessConfig = utimerscript { AppTick = 4 CommsTick = 4 } paused = false event = var=num_value, val="$(15)", time=0.25 event = var=num_value, val="$(9)", time=0.50 event = var=num_value, val="$(7)", time=0.75 event = var=num_value, val="$(5)", time=1.00 reset_max = nolimit reset_time = all-posted The "$(15)" notation above indicates that a 15-digit number is desired, but posted as type string. Recall the reason why we choose to pass the number as a string is that really big numbers are not able to be precisely represented as a double, as discussed in Section 2.2. To prepare for the next testing step, the test we will use to grade your lab, you can add one or more events to your script using known large prime numbers. 5.4 Grading: Test with Script Containing Large Primes In the test used for grading this lab, rather than leaving things to chance, we post some really big numbers that we know are prime, and then a couple small composite numbers. We should see that the small numbers don t get blocked by the big ones: 16

17 ProcessConfig = utimerscript { AppTick = 4 CommsTick = 4 event event event event event event event event event = var=num_value, val=" ", time=0.25 = var=num_value, val=" ", time=0.50 = var=num_value, val=" ", time=0.75 = var=num_value, val="125", time=1.00 = var=num_value, val="225", time=1.25 = var=num_value, val="325", time=1.50 = var=num_value, val=" ", time=1.75 = var=num_value, val=" ", time=2.00 = var=num_value, val=" ", time=2.25 } reset_max = 0 reset_time = all-posted Note also that we set reset max=0 so that the script is only executed once. To facilate testing, add plogger to your mission, in your primes.moos file. Make sure you add plogger to your Antler block with a line like: Run = NewConsole = false And configure plogger with its own configuration block: ProcessConfig = plogger { AsyncLog = true WildCardLogging = true WildCardOmitPattern = *_STATUS } LOG = 0 LOG = 0 Re-launch your mission and let it run for say a minute or so, which should give your app enough time to process all the incoming values, even the long ones. Then let s check the log file. Your mission should have created a folder like: MOOSLog_2_1_ _02_19/ This is where plogger generated all its output. The key file in here is the.alog file: 17

18 $ cd MOOSLog_2_1_ _02_19/ MOOSLog_2_1_ _02_19._moos MOOSLog_2_1_ _02_19.alog MOOSLog_2_1_ _02_19.blog MOOSLog_2_1_ _02_19.ylog Now you can see the sequencing of your application by grepping on the the PRIME RESULT variable: $ aloggrep MOOSLog_2_1_ _02_19.alog PRIME_RESULT NUM_VALUE Verify that the solutions to the three small numbers were posted quickly, almost immediately after their input was posted. Grading Criteria This lab will be graded on the specs given in Section 1.2. Additionally, the coding guidelines described in the following two C++ labs will be used to grade the quality of code used in your solution