Mapster: Application-Level File Sharing

Introduction

Project Deadlines

• There are 2 deadlines for this project, corresponding to the 2 checkpoints:
  1. Checkpoint 1: 9/27/04, 5:00pm
  2. Checkpoint 2: 10/7/04, 11:59pm
• You are strongly urged to start your project as early as possible!
• You must submit the relevant files (see below) for each checkpoint.
• Be aware that you don't have any slip day for homeworks and projects, for more information click here.

Instructions

• The project is to be done in C/C++ only.
• Read the lastest specifications carefully. Check periodically for updates. If you have any doubts, do not hesitate to contact your TAs.
• The tarred file proj1start.tar provides the following files and directores:
  1. include/mapster.h
  2. mapsterc.c
  3. Makefile
  4. client1
  5. client2
• mapster.h contains the constants defined in the specifications. Thus everyone has to use the same constants. DO NOT CHANGE THESE CONSTANTS.
• mapsterc.c provides a code skeleton for the client. You don't have to use this skeleton. It serves the purpose for helping you start your own implementation easily. Please also notice that mapsterc.c is incomplete comparing to the project specification. You need extend this skeleton, and modify any code to make your client work. For example, you might modify the parameters of functions for your own purpose. However, you should use global variables as less as possible.
• A Makefile is provided to help you bootstrap. Please go through the Makefile and be aware of what's going in within. You are also encouraged to write your own Makefile. This Makefile is for the sparc machines. If you want to use it on Linux, you might need to remove the "-lsocket" option in the LDLIBS line.
• client1 and client2 are two sample file directores of clients. Each of them contains a keyword file and a set of sample files. You don't have to use them, but your client is required to execute in the same directory as the one containing the keyword file and shared files..
• You can download the server program (mapsters) below and run it on your account to test your client.
• All header files are to be placed in the corresponding subdirectory include. For instance if you put your .c files in project1 then the relevant header files will be in project1/include.

Project Submission

• You are to submit the following files for checkpoint 1:
  1. mapster.h -- Mapster header file.
  2. mapsterc.h -- Mapster client header file, if you have it.
  3. mapsterc.c / mapsterc.cc / mapsterc.cpp -- Mapster client code file
  4. Makefile -- for compiling client program

  And for checkpoint 2:

  1. mapster.h -- Mapster header file.
  2. mapsters.h -- Mapster server header file, if you have it.
  3. mapsterc.h -- Mapster client header file, if you have it.
  4. mapsters.c / mapsters.cc / mapsters.cpp -- Mapster server code file
  5. mapsterc.c / mapsterc.cc / mapsterc.cpp -- Mapster client code file
  6. Makefile -- for compiling server and client program

  No other files must be submitted.

• To submit your files, create a directory, first copy the relevant files into that directory, making sure that the header files go into the corresponding include directory. Then go into that directory and then enter

  submit proj1checkpoint1

  or

  submit proj1checkpoint2

  for checkpoints 1 or 2 respectively.

• Please make sure that you submit the version you intend to submit. To be fair to everyone else we cannot accept cases where earlier versions were submitted in error. Note that you can submit as many times as you want, as long as the submissions are before the deadlines. We will grade the latest version (ie. the one closest to and before the deadlines).

Downloadables

• Specifications version 1.2 [ps, pdf]
  Mdification: incorporate important information from newsgroup; print out information for server.
• Specifications version 1.1 [ps, pdf]
  Modification: client pseudocode is more consistent with mapsterc.c than that in version 1.0.
• Server program binaries: you can download the server and use it to test your client program. Please check regularly in case we add additional materials to assist in debugging.
  1. Last update: 9/08, 10:30am
  2. linux binary: mapsters.linux
  3. sparc binary: mapsters.sparc
  4. mac binary: mapsters.mac

Modification to Project Specs

1. Comments that are not in proj1_specs_v1.2:
   1. Please remove any warning in compilation.
   2. After download, please publish only the new entry (keyword, downloaded file name). Publishing the whole keyword file after download is an example of bad design.
   3. Please use the SO_REUSEADDR option at your client and at your server. This is necessary for re-using the same port again after weird crashes. TAs will scan code whether you use SO_REUSEADDR properly.
2. Section 4.2 adds more information about how server should handle publish.
3. Section 6.3 specifies the print out information for both client and server. With verbose option, these outputs and only these outputs must be displayed to stdout.
4. Important comments from newsgroup:
   1. Section 5.2 adds more detailed description about message types and formats.
   2. Section 5.3 adds FAQs for implementation related to checkpoint 1.

Hints & Tips

1. Please read the specs carefully. Actions of server, message contents and formats between server and client are defined in the project specification. You have to follow the spec for the server implementation.
2. Here are some hints for server implementation. They are incomplete and might be sub-optimal. You don't have to follow them and you are encouraged to come out your own design.
   1. You can use two linked lists:
      1. One is for client list. It is used to track clients which have connected to the server. Each item of the client list is a struct containing the socket fd for the connection, the client's ip address and listen port number. You can use fd to uniquely identify a connection in this list
      2. Another is used to maintain the index table. Each item in the index table is a struct containing at least keyword, file name, IP address and listen port of the client storing the file. You can use IP address and listening port to uniquely identify a client in this list.
   2. The server should monitor activities on STDIN and all sockets. All fds, including STDIN, fd for server's listen socket and fds for clients (created by "accept"), should be put into fd set for select(). Every time when server creates a new fd using "accept", you need consider to put it into select().
   3. Whenever the server's listen socket detects anything via select, it means a Join request comes in. The server need to check the client list to see whether the same client has listened to the same port number.
   4. Let's think about how to handle publish. When a publish msg comes in, you know the fd from which it comes in. You also know the IP address of the client, the keyword and file name for the file. However, the listening port number of the client is missing. You need to go through the client list, and find out which port number the client is listening to using the fd. With consideration about duplicates, you can put (keyword, filename, IP, Port) in the index table.
   5. Handling search should be easy. You just need to handle exact match between keyword, and find out all files associated with the keyword.
   6. Handling leave and quit is simple. You need to remove the corresponding item for the client from both the client list and the index table. Recall you can use ip address and listen port number to uniquely identify a client.
   7. For print command, you are asked to print information from both client list and index table.
3. If you think there are problems please give all information you can obtain, although providing possible causes of error can sometimes be of help, it can also be misleading and confusing. It is important to provide as much information as possible. The reason is that what you think is wrong may not be the actual cause of the problem.
4. We will be covering the basics of socket programming during lectures and, if necessary, during discussions. Since the socket APIs have been around for such a long time, there are a great deal of sample code in the Internet, describing the basic operations that can be built from the APIs. It is to your advantage to read some of these sample code but know exactly what the code is doing and do not copy blindly. This is something which just about everyone has to do in the near future. The most important part of the project is knowing how to use the APIs in the application of networking concepts.
5. This website contains some sample code that might be of help. >

FAQs

• You need to handle crashes in your implementation. T here are at least two cases you need to handle:
  1. Client crashes while downloading. In this case, one peer (either sender or receiver) dies, another should not hang. It need move on and be ready for next operation.
  2. Server should notice that a client crashes. When this happens, it needs to remove from its index table all information related to this client.
• For the project implementation, we require you to use select() to multiplex I/O and socket operations together. select() has very nice properties and we want you to have a good exercise on it. You are not allowed to use multi-threading technique.
• If a client publishes a record (keyword, filename) multiple times, the server should only maintain one entry for this record in the index table.
• Test cases for checkpoint 1 are complete, and you might want to test your implementation using these cases. Test cases for checkpoint 2 are still under changing.
• We will test your implementation on the instructional machines. If you implement your code on Windows, you have to port your implementation and make it work on the instructional machines
• To maintain data structures either at the server side or at the client side, you can use generic library like STL if you want.
• Client port is specified via command line argument, which is stated on pg 5, version 1 of the specs.
• The server doesn't limit the number of clients it serves, and accept all incoming TCP connections. The server has to install some amount of client state when it receives the join request from the client. This is in the pseudocode on page 10, version 1 of the specs.
• When you to send a number, for example, port number, through the network, be sure you use htons(l) and ntohs(l); otherwise, the number will mess up, especially when the sender and receiver are running on different platform.
• A client should do a "Leave" before it quits the program. This will remove its states stored at the server; Otherwise it leaves the server in the inconsistent status and the server might give incorrect results in following operations.
• The server will never be aware that the client executed a bind. This information is not transmitted from the client to the server. Thus bind effectively makes no difference to the server.
• Also, it must be stated that we should not code our programs based on the server program, the starting point should be the specifications. Please do not attempt all possible combinations and see if that works with the server program given. If you implement the specs correctly then things will work. The specs is written based on the implementations of the client and server.

Testcases

1. Check all code paths mentioned in specs (Total 8pts):
   1. Basic correctness (Total 4pts):
      1. Join the network via the server (1 pt)
      2. Publish its shared files to server; see if the server gets them (1 pt)
      3. Search the server by keyword and have the client print zero, one, or multiple results based on the result message sent by the server (2pts)
   2. Based on the search results, download a file correctly from its peer. (Total 3pts)
      Make sure that:
      1. Error is handled if: 1)the peer is not listening; 2) a peer, either the sender or the receiver, crashes in the middle of downloading(1pt)
      2. Peer responds to download request by sending the correct file (1pt)
      3. The client correctly receives the file from its peer (1 pt)
   3. Leave and quit the network; see if the server is notified (Total 1 pt)
2. Basic robustness (Total 2pts):
   1. have the student's client download from each other a big file, say 200K, does everything work well (1 pt)
   2. several clients simultaneously download the same big file from a peer. do all clients get the whole file (1 pt)

1. Test your server by checking all code paths mentioned in specs: (Total 10 pts)
   1. Basic correctness: we will run 2-3 clients to test whether your server handles basic joining, publishing, searching and leaving operations correctly.
      1. Does server handle join and maintain listening ports of clients correctly? (2 pt)
      2. Is a publish sent by a client correctly stored at the server? (1 pt)
      3. If a client do publishing multiple times, does the server maintain the correct information and remove duplicates? (1 pt)
      4. Can the server handle search correctly? If there are multiple files associateed with the given keyword, are all results found and of search sent to the client correctly? (2 pts)
      5. Does the server correctly remove related information in the index table upon receive a "Leave" messsage from a client? (2 pt)
      6. Does the server handle the client crashes and remove all relevant information from the index table?(1 pt)
      7. Does the server print out information about clients in the system and the file index table correctly? (1 pt)
2. Test the client again using the same cases in checkpoint 1 (Total 10pts).
3. Test robustness of the implementation. To be coming. (5 pts)
4. Credit on the design and coding style of the project, including readability,modularization, clean error handling, memory free, etc. (5 pts)