Multimedia Networking Project 1b

Evaluation of Speech Detection Algorithm

Due date: February 14, 2002



You are to conduct experiments to evaluate the performance of your Speech Detection algorithm from Project 1. The focus of this project is not only on how the algorithm performs, but on the formulation of hypotheses; the design, implementation and analysis of experiments to test the hypotheses; and a writeup of it all.


Design experiments

In evaluating your algorithm, there are two measures of performance you will need to consider:

You will decide on how each is to be measured. For example, you may choose to get a user's opinion number from 1-10 to measure user perception and you may choose to use processing time to measure system overhead.

Then, you will manipulate the independent variable and determine its impact on the algorithm through your chosen performance measures. You must pick at least two independent variables. Some possibilities are:

In addition, you must chose 1 algorithm modification and evaluate it. Possibilities include:

You should formulate hypotheses about how a change in the independent variables affects your measures of performance.

Results and Analysis

You must provide details on both the results and the analysis. The results are the numeric measures recorded in the experiments, in the form of graphs, charts or tables. The analysis involves manipulating the data to understand relationships and interpreting the results. The analysis should consider whether the data supports or rejects the hypotheses.

Summarize findings

The main deliverable for this project is a report describing:


Good experimental writeups provide sufficient details for a knowledgeable reader to reproduce the results obtained. Keep this in mind when doing your writeup. In particular, for system measures, you should record details about the hardware and software used. For user perception measures, you should record background information on the subjects, familiarity with the topic, method sampled, etc.

Visualizations, such as graphs or charts, even simple ones, are typically much better representations of data than just tables of numbers. All graphs should include:

Graph Tips

If you are using Windows, MS Excel has good support for drawing graphs. You might try this tutorial to get started.

If you are using Unix, gnuplot has good support for drawing graphs. You might see for more information.

You might look at the slides for this project (ppt, pdf) and the slides for Experiments in computer science (ppt, pdf).

Hand In

You must turn in a hard-copy of your project report. Please include a title page with a title, abstract and group members. Unless otherwise specified, the hard-copy must be given to me, or delivered to FL138 on the day it is due.

Return to the Multimedia Networking Home Page

Send all project questions to the TA mailing list: