EE122 Project 1. Fall Version 0.4. Part A due at 11:50pm on Monday, October 11, 2010 Part B due at 11:50pm on Wednesday, October 27, 2010

Transcription

1 EE122 Project 1 Fall 2010 Version 0.4 Part A due at 11:50pm on Monday, October 11, 2010 Part B due at 11:50pm on Wednesday, October 27, 2010 Change History: 01 -> 02: Added clarifications to server requirements: must support 20 clients, must call on_client_disconnection, must send one JOB_ASSIGNED at a time. 02 -> 03: Added FAQ section, fixed inconsistency - 2 is a valid number in to be assigned. 03 -> 04: Updated the due date for part A. Introduction The galaxy is in great distress. Aliens have attacked the Earth and captured most of the Jedis. Surviving Jedis have gathered at the UC campus and are preparing the plans for a final strike back. However, to maximize the effectiveness of the defining battle they ask you to crack the alien encryption codes. Luckily, alien encryption, similarly to ours, is based on prime numbers and can be cracked if one has a large database of prime numbers. Your original plan was to employ the great millennium cluster, but after Jedi Arnold depleted UC s mineral reserves the cluster is under a constant risk of being shut down. Therefore, you decide to design a distributed system and employ whatever minerals are left throughout the Earth. Your approach is similar to SETI@Home project. You will write two programs: a client and a server. The server program will run at UC and you will distribute the client program to all the surviving human colonies that can contribute computational power. When the client program is started, it will connect to the server and send a LOGIN message containing the name of the colony. (You figured that if Earth survives, future generations would be interested in knowing how contributed to the victory.) When the server receives a LOGIN message from a client, it starts assigning jobs to it. A job is to check whether a large number is prime or not. Server assigns a job by sending a JOB_ASSIGNED message. When the job is done, client communicates the result back with a JOB_COMPLETED message. Usually, the server will respond with another JOB_ASSIGNED message, but sometimes it can be busy and respond later. To maximize its efficiency, the server can also send multiple JOB_ASSIGNED messages at once. Finally, you feel that more colonies will join the effort if you allow participants to communicate with each other. To enable the exchange of messages, you decide to add a SPEAK message. After the client logs

2 in, it can send SPEAK messages to the server containing its name and whatever message it wants. The server simply broadcasts the message to all connected clients (including back to the sender). In part A and B, you will implement the client and the server, respectively. The following sections describe the messages and the requirements in more details. Client/Worker Before going into details of client s operation, we describe the starter files available from the class web page. Starter Files --- messages.h --- We will be testing you client binary by looking at the message it sends and receives and at what it prints to stdout. To avoid the situation where you program is correct by you lost 20 points because you forgot a comma in your printf() call, we provided a number of functions that you should use to make sure your output is formatted correctly. Besides the printing functions, this file also contains two utility functions that you can use to check that the chat message and the name are well-formed. --- protocol.h --- This file contains the definitions of the structures that can be used to access message fields. Below is a usage example (without any checks): char buf[1024]; struct ee122_hdr *header; recv(sock, buf, 1024, 0); header = (struct ee122_hdr *) buf; printf( Message: ver=%d, type=%d, length=%d\n, header->version_, header->type_, ntohs(header->length_)); Message format is fixed. If you prefer not to use these structures, you must make sure that you messages look exactly as specified. --- constants.h --- This file contains some basic constants used throughout the project. --- client.c --- This file contains some basic setup for the client. You don t have to use this skeleton. It is provided only to help you get started. If you decide to start from the provided skeleton, you will essentially need to fill the places marked with TODO:. Note that the required code is substantial, so you will probably need to define and implement a good number of functions. For reference, our client implementation is around 450 lines (including extensive comments and blank lines). --- server.c --- This file contains some basic setup for the server.

3 --- server --- This is a binary of a reference server implementation. We provide it to ease your client development for part A. It can be started with./server -p <listening-port-number> -v where <listening-port-number> is the port number you want your server to listen on. Note that on many systems the lower 1023 numbers are reserved to programs that have root privileges. So, you should pick your port somewhere between 1024 and The -v flag enables verbose output. It can be useful for debugging. If you want, you can start the server without this flag. --- Makefile --- This is a make file that you can use to build your client and server binaries. If you are not familiar with make, check or any one of many make tutorial. To build the client, you can execute: make client To build the server, you can execute: make server To delete the generated binary files, you can execute: make clean --- a_client_test.py --- This is a testing script similar to the one that will will use to grade your submission. You can use this script to gain some confidence in your implementation. It should be placed in the directory containing the client binary named client. You can start it with:./a_client_test.py It will run 20 tests, print the progress as it goes, and the total number of points you got at the end. When a test fails, it will print output similar to: Running test_server_closes_connection - FAILED: Traceback (most recent call last): File "./a_client_test.py", line 575, in call_and_catch fun() File "./a_client_test.py", line 450, in test_server_closes_connection assert(clnt.start_client()) AssertionError From the output, you can see that start_client() failed in test_server_closes_connection(). If you want, you can look in to the testing script, but we won t support it. If you suspect that there is a problem with the script, think twice (or maybe thrice) before contacting your TA. Note that the script expects to receive certain output in certain time. We chose the timeout values with a lot of slack. If you think some timeout value is too short, contact your TA (after thinking twice :)). Client Requirement Because in the project, we will have another type of client, we will refer to the clients doing prime checks as workers. The connection between each worker and the server must be over TCP. The worker must be startable with the following command line:

4 ./client -p <server-port> -s <server-ip> -n <name> -v Where <server-port> is a port number that server is listening on, <server-ip> is the IP address in the dotted decimal representation (e.g special ip for the same machine ), <name> is the name of the client (that must contain at most 11 character, each either a letter or a number), and -v specifies that client should print out verbose output. When the client is started, it must attempt to connect to the server. If connection succeeds it must print Connected to server. to stdout (see on_connect_to_server()). Its first message must be the LOGIN message. This message must be sent only once during a run of the client. For each received JOB_ASSIGNED message, the client must respond with exactly one JOB_COMPLETED message containing mathematically correct result. If the client was started with -v option, before sending JOB_COMPLETED, it must print the result to stdout using the on_job_assigned() function. Note also that JOB_COMPLETED messages must be in the same order as the received JOB_ASSIGNED messages. The user must be able to input a chat message on stdin. The message is considered over when the user hits the enter key. The client is not required to read the input and send out a SPEAK message immediately. It is OK for the client to be blocked in recv() until the server sends some message. When recv() returns, the client must send out the SPEAK message (if user has entered a chat message on stdin) before sending any other message. Note that this implies that the client can be implemented using blocking sockets. You are obviously free to implement I/O multiplexing if you want. Whenever a client receives a well-formed SPEAK message from the server (including the message that it originally sent to the server), it must print it to stdout using the on_speak() function. A warning about implementing message receiving functionality. The recv() function returns as soon as some data is available and you don t know in what granularity the data will become available. You might receive a single byte or you might receive 10 messages at once. Your program must handle all these cases correctly. Client Error Handling Most of the functions in messages.h are related to different error cases. In this section, we detail what error cases we require you to handle. The cases are organized by the function you should call. on_client_connect_failure(): This function must be called if the client fails to establish a connection with the server. on_disconnection_from_server(): This function must be called when connection to the server is broken (not when the client closes the connection itself). This can happen for a number of reasons including the server crashing, closing the socket, sending a bad packet (violating TCP/IP spec), etc. on_malformed_message_from_server(): This function must be called server sends a malformed message. There are many possible ways a message can be malformed. These ways include: Server sends a LOGIN or a JOB_COMPLETED message. Server sends a message of an unknown type (different from LOGIN, JOB_ASSIGNED, JOB_COMPLETED, SPEAK)

5 Server sends a too long message (Our message are limited to 255 bytes including our packet header). Server sends a message of right type but of bad length (all messages but SPEAK have fixed lengths). Server sends a message with bad payload: 0 or 1 as the number in JOB_ASSIGNED. Non-printable character in name or msg of SPEAK. Non-null-terminated name or msg of SPEAK. Note that we provide functions check_message() and check_worker_name() to help with this case. After calling the appropriate on_*() function, the client must exit (wtih any return value). We don t attempt to fix the error in any way. There are many other error cases that we don t require you to handle. These include: Message is not 4 bytes aligned. Bad input from stdin: message too long, contains a null bytes somewhere in the middle. Bad command-line arguments. Server Requirements 1. The server program runs with the following program argument../server -p <listening-port-number> -r <first-num> where <first-num> is the first number that the server must assign to the client. Then, the server must assign the following consecutive numbers. If the server assigns a number and the client disconnects, the server does not have to keep track of this and reassign the number. In this project, we don t worry about storing the numbers and making sure that we have all of them. 2. The server must support at least 20 simultaneous clients. 3. When a client disconnects (whether itself or the server disconnects the client), server must call on_client_disconnection() with the name of the client. If the disconnecting client does not have a valid name (i.e. it did not login yet or the name in the login message is invalid), the server must call on_client_disconnection( invalid-name ). 4. The server must send one JOB_ASSIGNED message at a time. In other words, to be able to send n th JOB_ASSIGNED to a client, the server must have received (n-1) JOB_COMPLETED from this client. 5. The server must be able to bind the port and must not be affected by the TCP's 2*MSL constraint. In other words, the server must not give an error if restarted immediately after a crash. If bind() fails, the server must call on_server_port_bind_failure(). 6. After binding the port, the server waits for client connections without wasting CPU cycles (e.g. use select()). The server must never block - neither in recv() nor in send() nor in accept(). 7. Clients can log in using the LOGIN message. Other messages from the client prior to the LOGIN message must be ignored by the server. We assume that all clients will have different names (the server can behave arbitrarily if this assumption is violated). 8. Once logged in, if the client sends a LOGIN message, the server must ignore it. 9. The server must ignore JOB_ASSIGNED messages from clients. 10. The server must handle malformed messages from clients. A malformed message means any message that is normally handled by the server but that has a bad length of contains an invalid value. One exception to this rule is the JOB_COMPLETED message. The server does not need to check the mathematical validity of the result (it must check that the length is valid, is_prime is 0 or 1, and number is greater than 1). When server receives a malformed message, it must call

6 on_malformed_message_from_client() and close the client s connection. The server must handle all of the packets from the client up to the malformed one. There is one special case. If the server receives a LOGIN message with invalid name, it should call the on_malformed_message_from_client() function with parameter name set to invalidname. 11. The server must handle disconnection from clients. Clients can arbitrarily join and leave, and the server must not crash. 12. The server must not experience deadlocks and it must not block on a single client. 13. When the server receives a well-formed LOGIN message, it must call on_login() function. 14. When the server receives a well-formed JOB_COMPLETED message, it must call on_job_completed() function. The server does not need to check that the number it received was the last number it sent to the client. 15. When the server receives a well-formed SPEAK message, it must call on_speak() function and broadcast this message to all other connected clients. The server must not block in send() while broadcasting the messages. Server Implementation Hint Because you can never block, when select returns, you will call some recv() (and send()) but you don t know how much data it will accept. Thus, you might receive just one byte or 1.5 messages or 10 messages. If you don t receive a complete message, you will need to store the data you received somewhere and enter select() again. Next time, when there is more data available on the same input source, you need to recv() whatever is available and append it to the data you already had. When you get at least one complete message in this buffer you can handle it. The same also applies to send(). When you have a message to send out, you cannot just call send repeatedly until all the data is sent. You can only call send() once (if select() sets the corresponding socket as writable) and whatever data was not accepted you must buffer it and attempt transmission the next time this socket becomes writable. recv() when socket is readable > buffer > If there is a full message in buffer, handle it. Have a message to send > buffer > Drain (portion of) the buffer when socket is writable. Message Format As you know from the lecture about sockets, TCP sockets treat all messages as a continuous stream of data. To differentiate one of our messages from another, we define a message format. Each of our messages starts with a header. Our header is 32 bits in length and look like this: version (8 bits) length (16 bits) length (cont d) type (8 bits) version is the version of our protocol. It must always be set to EE122_VALID_VERSION, which is currently 4. length is the total length of the message in bytes including the header. type is the type of this message. Must be one of LOGIN, JOB_ASSIGNED, JOB_COMPLETED, or SPEAK.

7 All other messages start with this header and are padded to be 4-byte aligned where needed. Note that all numbers in the format are in network bytes order. LOGIN: version (8 bits) length (16 bits) length (cont d) type (8 bits) name (11 bytes) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) 0 (null termination) name is the name of the worker. If name is shorter than 11 bytes, unoccupied bytes must be set to 0. JOB_ASSIGNED: version (8 bits) length (16 bits) length (cont d) type (8 bits) number (4 bytes) number (cont d) number (cont d) number (cont d) number is the number whose primeness must be checked. Must be interpreted as unsigned 32 bit integer (you can use uint32_t type). Must be greater than 1. JOB_COMPLETED: version (8 bits) length (16 bits) length (cont d) type (8 bits) number (4 bytes) number (cont d) number (cont d) number (cont d) factor (4 bytes) factor (cont d) factor (cont d) factor (cont d) is_prime (1 byte) padding padding padding number is the number whose primeness was checked. Must be interpreted as unsigned 32 bit integer (you can use uint32_t type). Must be greater than 1.

8 factor is the smallest prime factor of number. Must be interpreted as unsigned 32 bit integer (you can use uint32_t type). If number is prime, factor can contain garbage and should be ignored is_prime is 0 if number is composite and 1 if number is prime. padding is extra space to make the message 4 byte aligned. SPEAK: version (8 bits) length (16 bits) length (cont d) type (8 bits) name (11 bytes) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) name (cont d) 0 (null termination) msg (variable length) name is the name of the worker. If name is shorter than 11 bytes, unoccupied bytes must be set to 0. msg is a variable length chat message. It must be null-terminated and the whole message must be 4 byte aligned. Getting Started To get started: Install VirtualBox Download the virtual machine from Note that the virtual machine is almost 1GB. So, you should probably download it while on campus. The extraction will take about 30 minutes. (It is this large because I have installed 300MB of security updates, building tools, and networking tools such as wireshark). Uncompress the vm and add it to VirtualBox. Choose New, then select Linux and Ubuntu. Then when asked for hard drive, choose existing and point to the unpacked.vdi file. Start the vm. The username for the vm is stud and password is network (without quotes). To run a command as root use sudo. Download the starter source code from the class s website and place it somewhere in the home folder (say proj1). You can do all of your projects in the vm. If you decide to do them on some other system, it is your responsibility to make sure your code compiles and runs on this vm. Untar the files (tar xzvf project1.tar.gz) and go into the project1 directory. Run make to build the files. You should not see any errors or warnings. Run the test script (./a_client_test.py). Since starter client is almost empty all tests should fail. Start designing and writing code! (You can use gedit, emacs (install with sudo apt-get install emacs23 ), vim (install with sudo apt-get install vim-gnome ), or whatever you want) As you make progress you can check your code either with the test script or by running a reference server binary. You are also welcome to use tools like tcpdump, wireshark, hping3, etc.

9 Hand-in Instructions You must submit a single tar archive through bspace for each of the two parts of the project. If you don t use any additional libraries and place all of you code in client.c and server.c files (let s call this the normal case ), the tar archive for submission can be generated using the following commands: STUD_ID= make handin_1a # for part a STUD_ID= make handin_1b # for part b where must be replaced by your student id. The first (second) command will generate project1a_ tar.gz (project1b_ tar.gz) file that must be uploaded. Besides the source code, the submission archive must contain Makefile and readme.txt files. In the normal case, you don t need to touch the Makefile and the readme.txt must be empty. If, however, you are using a non-standard library you provide instruction in readme.txt on how to install it in the virtual machine. Also, if you have implemented an additional feature, you must describe it in the readme.txt file and provide instructions how we can see the feature in action. After installing any additional libraries (if any), we will run make client to build your client executable for part A. We will run make to build both client and server for part B. If you choose to follow a path different from the normal case, you must make sure that these commands produce the client and server binaries named client and server, respectively. Policies We very strongly encourage you to work on the project with your peers. However, you must adhere to the collaboration policy outlined below. All source code must be written solely by you. You can discuss anything related to a project with anybody. If discussion happens in writing, it must not include code. You must not read other students code, with the exception of helping them debug a problem which they tried to, but could not, figure out themselves. You are welcome to read online tutorials and example code. However, you must not copy-paste significant (>10 lines) portions of code. We strongly encourage you not to use non-standard libraries and build the whole system yourself. It will feel much better :) and you will have learned the fundamental underlying primitives. However, we realize that some students might have had extensive experience in socket programming before and might like to spend their time learning something new. For such cases, you must contact us and ask for permission to use an additional library. In return, we might ask you to implement an additional feature. Be waned that you might use program similarity tests and/or tools like Google code search to identify breaches in collaboration policy. We have all been students and know how tempting it might be especially when the load of other classes is very high. To avoid such situations, and to actually learn a very useful skill, start the project early! Then, if you are stuck and your friends can t help you, come to TA s office hours. You can either bring a laptop, if you have one, or some way to get your files. TAs can put your files in a virtual machine and look at the problem with you. Grading

10 We will grade your submission with an automatic script similar to the one distributed with the source. The testing script will not be the same!!! We will change all the values and add more tests. So, programming just to satisfy the provided script is not a good idea. We reserve the right to change your score (either up or down) based on the manual inspection of the source code. We want you to have fun doing you projects. So, you are free to implement additional interesting features. If you do so, your features must not break any of the specified requirements. In particular, your code should still pass the test script. If you implement an interesting feature, you must describe it in readme.txt. You can earn up to an extra of 10% of the project s score with an additional feature. Project 1 is worth 20% of you final grade. Part A is worth 8% and Part B is worth 12%. An additional feature in Part A can earn you an extra 0.8% and an additional feature in Part B can earn you an extra 1.2%. Thus, the maximum possible score for project 1 is 22% of your final grade. FAQ 1. What should the client do if its command line arguments are bad? Answer: The client is not responsible for checking command line arguments. If arguments are bad, your client can behave arbitrarily. 2. I am running regular Ubuntu on my machine. Do I need to make sure my code compiles and runs on the VM? Answer: If you don t use a non-standard library and you did not modify your Ubuntu in any nontrivial way (changed the libc/gcc/make versions to some old or very new ones and using some new features in your code, patched your kernel, etc) your code should compile on the VM and you should not need to check it. HOWEVER, if you have any reason for doubt, do check it! All you need is install virtual-box, download and unzip the VM. If we will have to go back to you more than once, this can affect your grade. 3. What should the client/server do if send(), recv(), or socket() fail with a negative return value? Answer: Assuming your process behaves well (e.g. does not create thousands of connections), you are passing the right arguments, and your machine is sane, this should not happen. A correct client/server running on the VM will not cause it. If it is happening for you, you have a problem in your code - find and fix it. As a general good practice, if it happens, use perror() and exit your program. 4. Do we have to use the MAX_MESSAGE variable? Answer: You don t have to. It is there for your convenience. 5. When I call send(), it returns immediately. Wasn t it supposed to block? What does that mean? Answer: send() copies your data into a kernel transmit queue. If there is enough space in this buffer when you call send(), your data will be copied to this buffer and send() will return immediately. The kernel networking stack will be sending this data according to TCP rules. From the perspective of your application, once send() returns you don t need to worry about this data anymore - if both sides stay up it will be transmitted at some point. 6. Let s say a client received a JOB_ASSIGNED message from the server. While it was blocked in recv(), user typed a chat message. Should I send the SPEAK or the JOB_COMPLETED first?

11 What if the client typed two messages? Answer: You must send SPEAK first. You can assume that the user typed only a single message (has pressed return key only once).