Thesis Project


Design Subproblem Ordering Knowledge Elicitation


I don’t' want to write formal requirements and design documentation for this project but since requirements definition and design have to be done I might as well write down what I come up with.


Several things need to be determined in order to complete this project. One way that I like to start is to approach it from the perspective of the user by writing use cases. They are a good first step toward determining the requirements for the system. I have a lot of open issues. I have listed these at the end of each use case.


Use Cases:


Use Case 1: Knowledge Elicitation Session


  1. Obtain Subject Information: name, e-mail address, relevant experience (questionnaire?)
  2. Describe design problem to the subject. This is done using separately included descriptions, pictures, etc. of the problem. The subject can choose the information they wish to see by clicking on the appropriate links. These links should open a new browser window so that the information can be available while performing the KE task.
  3. The subject is given a type-in box in which they are asked to describe how they would perform the design task. They are also instructed to describe why they make their decisions.
  4. The subject is then asked to enter each design step taken, in order, they will also be given the opportunity to describe preferences and constraints at this time.
  5. After entering each step, the subject is presented with the steps, in groups of three. They are asked to list the similarities and differences between the steps. Specific dimensions that they will be using to compare include: what information is needed to complete the subtask, and what are potential problems that may occur when completing the subtask.
  6. The user is then presented with their order of steps for final review.
  7. The subject is then given the opportunity to modify their list of design steps by adding, deleting, or re-ordering.
  8. At the end, the subject is presented with another questionnaire, this one about the KE process. Questions should include opportunities for them to comment on the stimuli provided (so adjustments can be made) as well as the KE process.





Use Case 2: Data Analysis


For each subject:

  1. Get the final list of subproblems and their order.
  2. Look at the similarities/differences between subproblems to determine if any alternate ordering schemes exist (for example - can steps be done in parallel?)
  3. Did the subject make any changes to the list of steps after they went through the repertory grid exercise?
  4. Create a list of potential orders
  5. Create a list of questions


For the group of subjects:

  1. Is there a common order that they all chose?
  2. What are the agreements, disagreements
  3. From the combined results, create a list of potential orders
  4. Create a final set of questions or clarifications needed





Use Case 3: Experiment Construction


  1. Create a questionnaire file containing all questions that the subject will need to ask. These questions can be either short answer or multiple choice. The questions need to be input in a specific format so the experiment builder can use them to create the questionnaire.
  2. Create a usability file containing all the usability questions to ask the subject at the end of the experiment.
  3. Create a stimuli file containing html links for each of the stimuli for the experiment. Simuli need to be previously created and stored as html or gif files.
  4. Run the experiment builder to create the experiment. This program needs as input the name of the experiment, a brief description of the item being designed, and the names of the questionnaire, usability, and stimuli files. Additional configuration information may also be needed so it knows where to store everything.
  5. If necessary, copy the resulting html file(s) to the correct location.






Use Case 4: Verification Phase


  1. Prepare e-mail messages for each subject. This message should contain the set of subproblem orderings, and any questions that still need to be asked. The subject should be asked if they have any preferences for the orderings derived. Also, if the ordering specified by this subject did not make the final list, they should be told why and given an opportunity to agree/disagree.
  2. Send e-mail messages
  3. Process results. Any agreement? Disagreement? Why?




Implementation Details


Implementation will be a mixture of C, html, cgi, and Java. The goal is to come up with something that can be easily configured for different domains (and their stimuli). C, html, and cgi were chosen because I am familiar with them and feel comfortable completing the project in the time given. Some Java will also be used to that I can get more experience working with it. Most of the file I/O will be done in C because it's easier that way.


User Interface Design


A big advantage of web-based interfaces is that it's really easy to build display mock-ups. I haven't done this yet though.