CS 525s - Projects - Fall 2006

Information concerning the course project for CS525s is given below. The intent of the project is for students to select from a wide spectrum of possible projects depending on your interests and skills, as long as you can clearly demonstrate that there is some relationship to either the management of sensor or of streaming data. You are highly encouraged to take on a project in a topic that are not yet familiar with so that you assure you learn something new.

Teaming

You should work in groups of two students. In order to find team-mates for your project, you are free to prepare a short 2 minute presentation on your proposed project (to be given a few weeks into the course). But start thinking now ! You can also advertise your project idea to the cs525 mailing group to find potential partners.

Project Types

There are no restrictions on the 'type' of project you take on. Acceptable project types include:

• Software development of a particular subcomponent or software tool of a stream or sensor-management system. Such a development-oriented project should ideally be coded as an extension of an existing code base, be it CAPE, AURORA, or TELEGRAPH, or others, instead of a as a stand-alone system. Usage of existing software to work with specific hardware sensors, or simulations of sensors for that matter, would fall into this category. This should allow you to gain exposure to existing systems, and to experience some challenges faced with their extensions.

  Examples of this, to just name a few, include development of a buffer manager to support persisten storage for stream query operators, a distributed query allocator to support running a query on multiple machines, or new query operators not supported by the given system.

• Software development of some more open-ended idea or algorithms; which could be the realization of one or more algorithms or ideas you have identified in the literature into a working implementation. In some cases, it would be acceptable that such more open-ended coding is done as a stand-alone software prototype. Often, the implementation of two techniques or the additional of one technique into a system which already has at least one existing technique would be preferred, to allow for comparative testing of a set of techniques or algorithms against one another. Experimentation with your chosen algorithms to test their effectiveness, or extensions of the idea in some manner (to cover other cases, to drop any of its assumptions) is encouraged.

  Examples of this, to just name a few, include support of a new time semantics in an existing system, incorporation of a new join implememtation algorithm, of a new stream mining algorithm.

• Development of some full-blown application utilizing stream or sensor technology. Here, your focus would be to locate useful (real) data sources (say on-line weather feeds for the area or news reports and so on), to transform such data into a format that one of the existing stream engines can consume such feeds, and to constructs a variety of queries that would be meaningful for that application. Usage of an existing or development of a new application interface would be an optional plus, but not required. The main work would go into analyzing and understanding the stream features of your chosen applications, working with real data, and checking out the adequacy or lack of required features of current stream management prototypes.

  Examples of this, to just name a few, include support of some on-line monitoring of stock trends, analysis and reporting of bad weather, environmental monitoring, and so on. The of a new join implememtation algorithm, of a new stream mining algorithm.

• An acceptable project could also be the development of either an in-depth analsis of the available techniques in one particular 'subarea' within the scope of this course, and the development of a survey paper based on your literature search. Such a project can be completely paper-based (requiring no implememtation), as long as the final output is a substantial document that includes critical analysis of the literature, comprehensive coverage of the literature, as well as an annotated bibliography of the papers you have identified. Typically, I would expect that you have reviewed and understood on the order of 20 papers; or put differently, pretty much all relevant papers in the literature related to your chosen topic should have been identified - be it the new stream ones (there will not be many) or older traditional ones if those techniques happen to be still relevant.

  Note here that the focus of this survey must be different from the mini-tutorials that we will be writing as part of this class agenda, or if not, it must go significantly more into depth than what we tend to cover on a given subject in this course.

• Last but not least, open-ended research can also satisfy the course project requirement. In this case, the outcome would be a research paper that outlines the ideas you have reviewed from the literature, your analysis and critique of them, your newly proposed extensions, ideas and thoughts, and some justification for your proposed algorithm or ideas. Typically, some cost analysis of some case study illustrating that this idea could be potentially useful (faster, better in some manner) should be done.

  Potential directions for such mini-research could be found in most the current stream topics, given that the field is still fairly young and there are many low-hanging fruits. On possible way to attempt to start would be to take a technique from one paper/system and try to see how it could be applied to a different context/system-architecture. Another possible way to start would be to carefully think about the assumptions made in any of the papers that we will be reading, and then think about what it may take to break any of those. If it's trivial to drop an assumptions, then it is not a fruitful direction, but if it turns out not to be obvious how do overcome such a given assumption, then you may have found your topic of exploration. Third, future work descriptions at the end of papers are great sources for new ideas (though of course those authors themselves may be working on them as well). Lastly, thinking about different techniques in traditional database context, such as for example, transactions, recovery, multiple queries, and so on, and studying their application to this new stream context may lead to interesting observations

Project Stages and Deliverables.

The project has three stages:

• STAGE 0: Project Formation and Intent (Due: week 5)

  Each group will need to discuss with the instructor their ideas about the proposed project they are thinking about before the project intent submission. Ideally, each group should come with some clear ideas of what they may want to pursue, and possibly also what they do not not want to pursue. Finally, the written project intent would simply sketch out the overall type of project (see categories listed above) and expected direction. A single page in length defining the project may be sufficient, however, any length is acceptable. The instructor will provide you with a go, no-go, or other feedback.

  Note: Please discuss your ideas on a project with me before finalizing and submitting your proposal. Your project intent will be approved, disapproved or some modifications of direction will be suggested, but you will not receive a grade for this stage zero.

• STAGE 1: Project Proposal (Due: week 7)
  

  Each group will turn in a typed proposal document (about 2 to 5 pages) defining the finalized project. After this, there is no turning back! At this time, you are encouraged to already have deeply thought about your proposed work, and also solved some initial part of it. Definitely, make sure to have dug in deep enough to understand the feasibility of your proposed direction, and/or to have looked at the necessary background knowledge and skills needed to pursue the project. This proposal should clearly explain the proposed work to be done, and also list the relevant environment (tools) you have set up or papers you have read in order to succeed to complete the project. Do provide an expected schedule for your planned completion, including a list of tasks to be undertaken week by week, and the deliverables for the end of the course.

  Note: This will count for 10% of your final project score.

• STAGE 2: Progress Report (Due: week 11)

  This progress report should clearly state the current status of the project. This report must be typed and typically is between 5 to 20 pages long. By this time, I would expect you to be already half way towards completing your project. I would expect you to have conducted all necessary background work including establishing a bibliography and reading relevant manuals and literature, installing and testing all necessary software, resolving specific design issues, and refining the project plan or possibly re-directing the effort based on your background studies. Make sure to include literature you have reviewed for the project, all other tasks you have accomplished as well as your basic idea of how you are approaching the problem. You also need to construct a precise schedule and task list as what remains to complete the project.

  Note: This report will carry 30% of your final project score.

  The most important part here will be that you demonstrate that you have completed a sufficient amount of work towards completion of the project (you are expected at this stage to be about half way through your project). Also, there needs to be a clear plan of what will be accomplished by the end of the project. See more guidlines below.

• Stage 3: Final Project Presentation and Report (Due: week 15 (last day of semester))

  The final project presentation and demonstration will be done in class by each team during the second to last class session. We'll determine the exact time allowed per project once we know how many different projects we have on-going. See below for what aspects should be stressed in this presentation.

  This presentation must be supplemented with a final project report due at the same time. This report should be a well-written technical report describing your project (and put on the web). It is likely that this report will essentially be an extension of your progress report. Depending on the nature of your project, it will need to contain detailed designs, system architectues, key technologies used, experimental charts, sample runs, a detailed analysis of the results, a system design, a justification and evaluation of selected tools, etc. The report typically is between 10 to 20 pages.

  Note: This part of the course project will count towards 60 % of the project score.




---

Grading of Project Progress

The grade of this assignment will be given using the following as guideline:

• the difficulty and size of the project you have chosen to work on,
• your overall solution approach and techniques utilized,
• the progress made towards your project thus far (i.e., are you about half-way into the project by now?)

• overhead slides you use to give your oral project presentation (are they well-organized, clear, and professional quality?)

• the oral project presentation itself (is it clear, understandable and informative?)

• the project report that you turn in to me in hard-copy; they can be a superset of the ones you used to give the oral project presentation, if you want, and they can contain other appendices (such as example runs or whatever), if you want. (is it well organized, complete, and informative?)

• the time plan for the remainder of the project (do you now have set up a clearn and realistic plan of what tasks need to be done during the remaining 6 weeks of the project in order to complete it in time by end of semester ?)

• the understanding and contribution of each team member of his or her part of the project, as well as of the overall group product. YOU ARE REQUIRED TO APPEND ONE PAGE TO THE END OF YOUR MATERIAL YOU TURN IN THAT STATES THE PERCENTAGE OF CONTRIBUTION FROM EACH TEAM MEMBER INTO THE PROJECT EFFORT THUS FAR. YOU CAN SIMPLY SAY HERE THAT EQUAL EFFORT WAS PUT FORTH, IF YOU WISH, OR YOU CAN LIST THE TASKS DONE BY EACH MEMBER. [no pts, except this may be used to not assign the same score to each of the team members. the default is all get the same score, unless there is some inequality of effort.]



---

Project Grading (general)

The final grade of the team project (which may not necessarily be the same for each member of the team) will consist of:

• the difficulty of the project you have chosen to work on, and your solution approach,
• the oral project presentation (covering the problem your group tackled, your solution approach, and your results at the end of the course),
• the written documentation of your project in the form of a report (which you will have to enter on the Web to make it accessible to other students in the class),
• the demonstration of your system (including successful example runs), only if applicable and
• the understanding of each team member of his or her part of the project, as well as of the overall group product.



---