_,''"`. ,~
("`-''-/").___.- ( )_ \ |
`6_ 6 ) `-. `_ ).`--.' /
(_Y_.)' ._ ) ', . ``-..'
`--' _.- _/ /--'_.' ,'
(il) (li),' ((!.-'
The term is only halfway over, but your Distributed Computing Systems course has already been brutal - you need some R&R if you are to make it to the break. What you want to do is kick back, relax and watch a flick from the comfort of your home. Using your new found networks knowledge, you have a plan to make this happen - the magic of the some text-based video, your computer and, of course, some slick network programming!
Due date: Sunday, February 14th, by 11:59pm
You will design, develop and test Nutella, a light-weight, peer-to-peer streaming movie system. Nutella uses multicast for efficient decentralized queries and text-based movies for lightweight videos that can be easily created, transferred and decoded, as well as constructed by hand.
Nutella uses a non-centralized system for movie searching. Users input a text-based query, which the local Nutella client multicasts to other Nutella peers. Any Nutella peer with a match on the content responds. At that time, the local Nutella client contacts a remote peer that responded and requests the movie. The remote peer then streams the movie to the local Nutella client.
From the viewpoint of an active Nutella client looking for a movie, the steps of the search process are as follows:
From the viewpoint of a passive Nutella client (the Nutella "server" process), the steps are:
When contacted, a Nutella peer streams a requested movie. Whereas downloading and playing a movie means the entire movie file is transferred to a client before it is played out, streaming has a client play out the movie while it receives the frames over the network. So, when streaming, the server sends the movie frames at the frame playout rate.
Nutella streams at no more than 10 frames per second. Streaming at
1-3 frames per second is fine, too (and it means the movie needs fewer
frames for the same amount of viewing time :-)). In Linux C/C++,
fine-grained delay between sending each frame can be achieved by
using nanosleep() (see the man pages for details). While
the actual frame rate is not particularly important for this
assignment, the idea is to get an "animated" effect.
You can assume a Nutella peer only sends one movie at a time. Thus, while sending a stream, the Nutella peer can either ignore any subsequent multicast searches for movies, or ignore any additional requests for streaming movies. Adding the capability of simultaneously streaming more than one movie (to different Nutella peers) is optional.
A user should be able to play a movie (i.e., request a stream from a different Nutella peer) while sending a movie to someone else. Note, this requirement can be handled by actually having a multi-process/threaded Nutella client or by having the Nutella player being a separate program/process from the Nutella streamer.
Each Nutella peer has some movies they are willing to share by streaming to other Nutella peers (you can assume some tit-for-tat reasoning for this, not specified in this assignment). For convenience, the list of movies a peer has should be specified outside of the compiled code (i.e., not hard coded). Options include: 1) kept in a configuration file (say, ".nutella") which contains information on the file location of each movie, or refers to a directory with the movies; 2) an environment variable that refers to the movie directory (e.g., NUTELLA=/home/claypool/nutella); or 3) specifying the movie directory via a command line argument, where every file in that directory is a movie.
The movie files themselves are human-readable text containing the animation frames and a frame separator. The format of a movie file is:
picture 1, arbitrarily long end picture 2, arbitrarily long end ...
For example, the start of the walk.txt file is:
o
|
|
end
o
|
/ \
end
o
|
|
end
o
|
/ \
end
o
|
|
end
...
Earlier creative creations include:
For inspiration, there have been some amazing, hand-made creations of ASCII movies, such as Star Wars ASCIIMation. There is also a community that has worked on converting and playing standard image video as ASCII video, such as at ASCIImeo.
Feel free to be creative with the movies you make. If you stick to the movie format given above, others in the class can share your creations. Have fun!
Nutella uses IP multicast sockets for query communication. IP
multicast allows a single message sent to potentially reach multiple
receivers. An end-point of a multicast socket is the same as that of
a unicast socket, specified by an IP address and a port. The valid
range of multicast addresses are: 239.0.0.1
to 239.255.255.255. The valid range for ports are
integers above the ephemeral port range (so, 5000+).
To simplify the use of multicast sockets, you may use some code
wrappers (think of them as middleware) written especially for projects
like this. The source file containing these wrappers
is msock.c and a header file
containing prototype definitions
is msock.h. The routines
provided in msock.h are:
/* msockcreate -- Create socket from which to read. return socket descriptor if ok, -1 if not ok. */ int msockcreate(int type, char *address, int port); /* msockdestroy -- Destroy socket by closing. return socket descriptor if ok, -1 if not ok. */ int msockdestroy(int sock); /* msend -- Send multicast message to given address. return number of bytes sent, -1 if error. */ int msend(int sock, char *message, int len); /* mrecv -- Receive message on given mcast address. Will block. return bytes received, -1 if error. */ int mrecv(int sock, char *message, int max_len);
To use the msock wrappers, you must compile the file
msock.c yourself. Do this by downloading both the
msock.c and msock.h files to your directory
and using the command gcc -c msock.c. Better still, use
a Makefile.
See mcast.tar for an example
of how these wrappers might be used (extract with tar xvf
and then type "make").
You may only need one multicast address for your Nutella system, since both queries and responses can be to the same address. However, conceptually it may be cleaner to have a multicast address for queries and a multicast address for responses. For example:
USE IP PORT Query 239.xxx.xxx.1 7000 Response 239.xxx.xxx.2 7001
Warning! Be careful of errant multicast code. Below is an example of what might happen due to multicast program that sends too much data (this one due to my own errant code :-().
From: "Charles R. Anderson" <cra@WPI.EDU> To: system@cs.wpi.edu, mvoorhis@WPI.EDU, jbanning@WPI.EDU Cc: netops@WPI.EDU Subject: cs.wpi.edu flooding network cs.wpi.edu is flooding the network with multicast traffic, causing network outages in Fuller Labs. I have disabled the port. -- Charles R. Anderson cra@wpi.edu Network Engineer (508) 831-6110 Computing and Communications Center X2220 on-campus Worcester Polytechnic Institute Fax (508) 831-5483In general, you should be ok if you only send one multicast packet request for each query the user types in but be sure to debug carefully before you run multicast code and monitor your code performance as your do so.
Important! The network port for the streaming server can have default values, but must be also configurable via a command line option (or something similar).
Below is an example of a possible peer-to-peer interaction, with one node acting as a server and the second a client:
SERVER PEER CLIENT PEER
Server started. Client started.
Listening... Enter movie name: Thor
Sending search request
Waiting for response ...
Received search request for "Thor"
Movie not here, so no response
Listening...
Timeout, movie not found
Enter movie name: Walk
Sending search request
Received search request "Walk"
Movie here
Sending IP 130.215.36.67, port 7832 to client
Listening...
Received IP and port from server
Sending request to stream...
Connection request received.
Streaming movie "Walk" to client... O
Movie done. Sending done message. |
Listening... / \ ...
Received done message.
Enter movie name:
If you want to make a player that reads from a movie file, your code would:
Open file
While file not empty
Read everything until "end", writing to stdout
Discard "end"
Pause appropriate amount
Clear screen
end while
Close file
Making such a player can be good for trying out movies before offering
them for streaming or as part of implementing a multicast client
player.
For your streaming socket code, use the information on sockets from your project 2, and the class slides on sockets.
Remember, if a process reads from a socket that has no data, it
blocks until there is data. You can poll a non-blocking socket for
data (use MSG_DONTWAIT with recvfrom() or
use fcntl() on the socket). Or, you can
use select() to see if any socket has information.
See select.c for a
C/C++ sample and do a man 2 select for more information.
Or, the ioctl() call can be used to see the number of
bytes available in a socket before reading using the FIONREAD
parameter as the second argument (e.g., ioctl(sock, FIONREAD,
&bytes)). Again, see the man pages for more information.
To clear the screen between frames, check out this snippet of code (works on most terminals).
After designing, implementing and testing your Nutella streaming system, you conduct some experiments to determine the network data rates required by a working system and predict the number of users supported.
First, determine the data rate required for a movie query-response protocol and streaming of a typical text-based movie. This can be done by measuring network load using a tool such as Wireshark. But better, is to determine network load based on analysis of your code and the movie content. In other words, you know the sizes of your queries and responses, as well as the sizes of videos and their frame rates. You can use this information to determine network data requirements. Make sure to consider lower-layer header sizes!
Assuming users continually watch movies (query-watch-repeat), compute how many users Nutella could support on a typical LAN. Be sure to explicitly state your assumptions about movies (length, frame rate, frame sizes) as well as assumptions about the LAN. Provide references, where appropriate.
Assuming Nutella did multicast streaming in addition to multicast queries, compute how many users Nutella could support on a typical LAN.
Repeat the same analysis for actual (non-text-based) movies (e.g., NetFlix) on a typical U.S. residential broadband network. Be sure to provide data on movie data rates, with references, as appropriate.
When complete, prepare a brief (1-2 page) write-up with the following sections:
Important! Before you are ready to submit, make sure your
nutella system works on the CCC machines (e.g., one peer
on cccwork2.wpi.edu and another
on cccwork3.wpi.edu and a third
on cccwork4.wpi.edu). Regardless of where you develop
your code, you are responsible for making sure your projects run on
these machines!
You must hand in the following:
.h files.
README.txt that contains information required to run
your system.
Also, be sure to have your experiment writeup in a clearly labeled file. The document must be in pdf format.
Before submitting, "clean" your code (i.e., do a "make clean") removing the binaries (executables and .o files).
Usezip to archive your files. For example:
mkdir lastname-proj3 cp * lastname-proj3 /* copy all the files you want to submit */ zip -r proj3-lastname.zip lastname-proj3 /* package and compress */
To submit your assignment (proj3-lastname.zip), log
into the Instruct Assist website:
https://ia.wpi.edu/cs4513/
Use your WPI username and password for access. Visit:
Tools → File Submission
Select "Project 3" from the dropdown and then "Browse" and select
your assignment (proj3-lastname.zip).
Make sure to hit "Upload File" after selecting it!
If successful, you should see a line similar to:
Creator Upload Time File Name Size Status Removal Claypool 2016-02-10 22:40:03 proj3-claypool.zip 1508 KB On Time Delete
A grading guide provides a point breakdown for the individual project components. A more general rubric follows:
90-100 The streaming system meets all specified requirements. Nutella p2p streaming fully implemented, with clients able to query movie, server processes responding, clients playing and the system being robust overall. Proper use of multicast and socket communication. All documentation included. Experiment report is clear, complete and includes all needed information.
80-89 The streaming system meets most specified requirements. Nutella p2p streaming implemented, with queries, response and movie playing, but occasional bugs and/or documentation is missing. Experiment report is complete and includes all needed information.
70-79 The streaming system meets some specified requirements. Nutella p2p streaming framework setup, but query or response not fully functioning and playout of movie may not commence properly. Occasional bugs, but system still compiles and runs. Documentation may be missing. Experiment report is incomplete and/or is missing some needed information.
60-69 The streaming system fails to meet many specified requirements. Nutella p2p streaming not functional. Server and client pieces compile and run but are not fully integrated. Socket communication works for some aspects of the system but not completely. May be some bugs. Documentation may be missing. Experiment report is incomplete and is missing some needed information.
0-59 The streaming system does not meet most specified requirements. Nutella p2p system not implemented in that most of the system cannot be started, significant problems or crashes arise during even short runs. Documentation missing and/or problems in compilation. Experiment report is mostly incomplete or is missing.
Return
to 4513 Home Page
Send all questions to the TA mailing list (cs4513-staff at cs.wpi.edu)
.