Shell Project, part 3 (with Buddy System Memory Manager) ( points)

Transcription

1 CS 453: Operating Systems Project 6 Shell Project, part 3 (with Buddy System Memory Manager) ( points) Due Date -On Class Home Page 1 Introduction In the last release of the mini-shell we will provide our own memory management. More specifically, we will replace malloc with your own memory management scheme based on the Buddy system discussed in class. 2 Startup Run the git pull --rebase command in your backpack folder (on the master branch). You should see a p6 project folder in your backpack with all the starter code. It should look like the listing below: [amit@localhost backpack(master)]$ ls -R.: backpack.sh buddy-preload Buddy-System-notes.pdf buddy-non-preload buddy-system-movie.mpg Makefile./buddy-non-preload: buddy.c buddy.h buddy-test.c buddy-unit-test.c Makefile malloc-test.c./buddy-preload: buddy.c buddy.h Makefile malloc-test.c [amit@localhost backpack(master)]$ First, you will be completing the buddy.c and buddy-unit-test.c files in the buddy-non-preload subfolder. Make sure to study the header file buddy.h for the javadocs of functions that you have to implement. Then you will port the over to the buddy-preload folder. The porting only involves changing some function prototypes.

2 3 Requirements 3.1 Buddy System Memory Management (70 points) Implement your own memory manager using the Buddy Algorithm. You should use the sbrk() to initially allocate a large block of memory. A good initial amount is 512MB. See the example memory-management/sbrk-test.c for the usage of the sbrk function. From there on manage the chunk of memory returned by sbrk using your own memory management functions. Note that you will have to store the data structures used to represent the buddy system some where in memory. There are two parts to this issue. First, you need to store the array of pointers to the blocks somewhere. You may reserve some amount of memory in the chunk you obtained from sbrk for this purpose. Or you may statically declare these pointers in you code. The second option is simpler. For this purpose you may assume that the maximum amount of memory that you will ever manage is limited to 32GB. Please see the starter code for sample declarations using this option. Secondly, the pointers associated with each free block in the buddy system should be stored in the block as explained in the Buddy system algorithm. Have all the initialization be in a separate function. If the user doesn t call this function, then it is transparently called whenever the user calls buddy malloc/calloc for the first time. You can test for that using a static variable. We will use the following prototypes for the buddy functions: void buddy_init(size_t); void *buddy_calloc(size_t, size_t); void *buddy_malloc(size_t); void *buddy_realloc(void *ptr, size_t size); void buddy_free(void *); Note that if a 0 is passed as an argument to buddy init, then it initializes the memory pool to be of the default size (512MB, as specified above). If the caller specifies an unreasonably small size, then the buddy system may not be able to satisfy any requests. That is acceptable behavior. If the memory cannot be allocated, then your buddy calloc or buddy malloc functions should set the errno to ENOMEM (which is defined in errno.h header file). Note that we have provided you with a buddy.h header file that contains all the above declarations. We have also provided a skeleton buddy.c that has the declaration for the pool and the table of lists for the buddy system. 3.2 Testing (10 points) Build a test suite for your buddy system. Sample test code that you can use as a starting point is in the buddy-non-preload/ folder. It contains two performance test files: buddy-test.c and malloc-test.c that run identical tests using the two different allocators. You should not modify these two files! In addition, there is a unit test file buddy-unit-test.c, to which you are required to add more tests. 2

3 3.3 References Read Section 5.4 (Address Arithmetic) and Section 8.7 (Example A Storage Allocator) from the C book by Kernighan and Ritchie to help you get started. Donald Knuth. Fundamental Algorithms. The Art of Computer Programming 1 (Second ed.) pp Addison-Wesley. Look up Buddy Memory System on Wikipedia. 3.4 Interposing and Integrating (20 points) Interposing Malloc Look at the example in memory-management/library-interposer to see how you can interpose all your malloc/free/calloc calls. This allows us to even get readline to use our malloc implementation. To use interposing, your buddy system allocator will need to implement the same interface as malloc (and with the same function names). We will make our buddy system into another shared library, which will be named libbuddy.so. Use the following command to interpose for malloc using libbuddy.so: LD_LIBRARY_PATH=.:../p0/lib LD_PRELOAD=libbuddy.so./mydash The command says to search for the preloaded library in the current folder (we also included../p0/lib so the program can find the libmylib.so library). However if the mydash program calls chdir system call, its current directory changes to something else and the system will no longer be able to find the buddy library. The solution to that is to give the full path of the current folder: LD_LIBRARY_PATH=/home/faculty/amit/cs453/buddy/:../p0/lib LD_PRELOAD=libbuddy.so./mydash but that would be different for everyone so a neat solution is to use the pwd command to get the fullpath of whatever the current directory is where the buddy library is located. LD_LIBRARY_PATH=$(pwd):../p0/lib LD_PRELOAD=libbuddy.so./mydash The command $(pwd) runs the command pwd and returns the output as the right hand side of the assignment. Using backquotes as in pwd does the same thing. Note that to time the interposed version, use the following command: time LD_LIBRARY_PATH=$(pwd) LD_PRELOAD=libbuddy.so./malloc-test <appropriate arguments> 3

4 3.4.2 Integrating with your shell Use the interposing version of your buddy system to integrate with the shell. Run all the base tests on your dash to check that it works well with the buddy memory allocator Integrating with other programs Now you can use preloading to test your buddy system with any program on your system! Be warned though that this may not work as most programs are multi-threaded and your buddy system isn t (unless you do the extra credit). You would also need to initialize the buddy system to have more memory (1-2G or more). This part is optional! 3.5 Extra Credit: Buddy System Performance (10 points) Test your buddy system implementation against malloc and make sure that it out-performs it. For the purposes of measuring performance, use the buddy-test.c and malloc-test.c code provided in the sample code for this project. Here is performance comparison for the reference solution (tested on onyx, compiled with -O optimizer flag) [amit@onyx buddy-system]$ time buddy-test s real 0m1.817s user 0m1.812s sys 0m0.001s [amit@onyx buddy-system]$ time malloc-test s real 0m2.838s user 0m2.830s sys 0m0.001s [amit@onyx buddy-system]$ 4 Extra Credit: Thread-safety (10 points) Make your buddy system library be thread-safe. Compare its run time performance against malloc and report in your README.md file. 5 Hints on Testing and Development. Implement the Buddy system as a separate class and test it using small programs. A good simple test would be to use your linked lists and the TestList program with your memory manager. Once you are confident in your Buddy memory manager, then integrate it into your 4

5 shell and then test the shell again to make sure that it still works as well as before. Your test cases from the previous version of the shell will come in handy here. 6 Documentation (20 points) Use your README.md file to communicate the structure of the files, the significant highlights of your project, test cases, the limitations and features, things that can still be improved etc. Continue to use doxygen tool from the previous project. 6.1 Source Code Control Include a section in your README.md file with your observation on your usage of Git. 7 Project Layout Failure to follow these instruction on the directory layout will cost you at least 10% of the grade. The top level of your submission directory for this project must have a Makefile that builds the both the preloaded and non-preloaded versions and other related text programs. The structure of the folder should be the same as what was given to you via backpack. Make sure to not modify the two test programs buddy-test.c and malloc-test.c. To test the buddy system library with the mydash project, you can simply copy your buddy system library to your p3 project and preload it to test it with your mydash project. This is how we will test your project. Make sure that your p3 project is up to date so that when you make a branch for this project, we will have the latest version of p3 project on it. Also make sure that there is a README.md file on the top-level that contains your name, date, assignment number, design and observations about the assignment, and test plan. It should clearly document what parts of this assignment you have attempted. Prepare your directory for submission by removing all executables and object files and other clutter. You should only have the source code, README.md file, Makefiles, test scripts and test files before submitting. 8 Submitting the Project Open up a terminal or console and navigate to the backpack folder for the class. Navigate to the subdirectory that contains the source files that you worked on for the project/homework. Suppose that we are submitting three files README.md, main.c and utility.c and one folder named include. 5

6 Clean up your directory make clean Remove any other unnecessary files that were generated by you or the auto-grader. Add all your changes to Git (this would be specific to your project) git add main.c utility.c include/ README.md Commit your changes to Git git commit -a -m "Finished project p6" Create a new branch for you code git branch p6_branch Switch to working with this new branch git checkout p6_branch (You can do both steps in one command with git checkout -b p6_branch ) Push your files to the Backpack server git push origin p6_branch Switch back to the master branch git checkout master Here is an example workflow for submitting your project. Replace p1 with p6 in the example below. [spanter@valkyrie shane(master) ]$ git checkout -b p1_branch Switched to a new branch p1_branch [spanter@valkyrie shane(p1_branch) ]$ git push origin p1_branch Total 0 (delta 0), reused 0 (delta 0) To git@nullptr.boisestate.edu:shane * [new branch] p1_branch -> p1_branch [spanter@valkyrie shane(p1_branch) ]$ git checkout master Switched to branch master Your branch is up-to-date with origin/master. Help! I want to submit my code again! No problem just checkout the branch and hack away! [spanter@valkyrie shane(master) ]$ git branch -r origin/head -> origin/master origin/master origin/p1_branch [spanter@valkyrie shane(master) ]$ git checkout p1_branch 6

7 Branch p1_branch set up to track remote branch p1_branch from origin. Switched to a new branch p1_branch [spanter@valkyrie shane(p1_branch) ]$ touch foo.txt [spanter@valkyrie shane(p1_branch) ]$ git add foo.txt [spanter@valkyrie shane(p1_branch) ]$ git commit -am "Adding change to branch" [p1_branch 1e32709] Adding change to branch 1 file changed, 0 insertions(+), 0 deletions(-) create mode foo.txt [spanter@valkyrie shane(p1_branch) ]$ git push origin p1_branch Counting objects: 3, done. Delta compression using up to 4 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (3/3), 286 bytes 0 bytes/s, done. Total 3 (delta 0), reused 0 (delta 0) To git@nullptr.boisestate.edu:shane 36139d1..1e32709 p1_branch -> p1_branch [spanter@valkyrie shane(p1_branch) ]$ git checkout master Switched to branch master Your branch is up-to-date with origin/master. [spanter@valkyrie shane(master) ]$ We highly recommend reading the section on branches from the Git book here: Git Branches in a Nutshell 7