I don’t' want to write formal requirements and design documentation for this project but since requirements definition and design have to be done I might as well write down what I come up with.

Several things need to be determined in order to complete this project. One way that I like to start is to approach it from the perspective of the user by writing use cases. They are a good first step toward determining the requirements for the system. I have a lot of open issues. I have listed these at the end of each use case.

1. Obtain Subject Information: name, e-mail address, relevant experience (questionnaire?)
2. Describe design problem to the subject. This is done using separately included descriptions, pictures, etc. of the problem. The subject can choose the information they wish to see by clicking on the appropriate links. These links should open a new browser window so that the information can be available while performing the KE task.
3. The subject is given a type-in box in which they are asked to describe how they would perform the design task. They are also instructed to describe why they make their decisions.
4. The subject is then asked to enter each design step taken, in order, they will also be given the opportunity to describe preferences and constraints at this time.
5. After entering each step, the subject is presented with the steps, in groups of three. They are asked to list the similarities and differences between the steps. Specific dimensions that they will be using to compare include: what information is needed to complete the subtask, and what are potential problems that may occur when completing the subtask.
6. The user is then presented with their order of steps for final review.
7. The subject is then given the opportunity to modify their list of design steps by adding, deleting, or re-ordering.
8. At the end, the subject is presented with another questionnaire, this one about the KE process. Questions should include opportunities for them to comment on the stimuli provided (so adjustments can be made) as well as the KE process.

• Step 5 could take forever if there are a lot of steps. This will drive the domain expert nuts. Is there a way that this can be streamlined? Limit the number of steps or increase the size of the groups?
• The thesis proposal also wanted step 5 repeated after changes are made to the list of design steps. This will be very time consuming. I would prefer to skip that step but maybe prompt the subject for an explanation of why they made their changes. I'm not sure how to work it if the subject decides to completely re-do their list though…
• Should a pause feature be included so the subject can stop a session in the middle and resume later?
• I know how I want to lay this out but I need to figure out how to do it in html! (and Java…)

For each subject:

1. Get the final list of subproblems and their order.
2. Look at the similarities/differences between subproblems to determine if any alternate ordering schemes exist (for example - can steps be done in parallel?)
3. Did the subject make any changes to the list of steps after they went through the repertory grid exercise?
4. Create a list of potential orders
5. Create a list of questions

For the group of subjects:

1. Is there a common order that they all chose?
2. What are the agreements, disagreements
3. From the combined results, create a list of potential orders
4. Create a final set of questions or clarifications needed

• How is all this data being stored?
• Any way to automate part of this process?
• If subjects are allowed to specify the steps themselves, there is likely to be a lot of disagreement. They may approach the problem at different levels of granularity. This is a good argument for not letting them pick the steps, just the order!
• How much abstraction of subject steps should be performed in order to be able to meaningfully combine results? This will be particularly difficult if the domain chosen is one that is unfamiliar to me. I might think two steps do the same thing when they are completely different!

1. Create a questionnaire file containing all questions that the subject will need to ask. These questions can be either short answer or multiple choice. The questions need to be input in a specific format so the experiment builder can use them to create the questionnaire.
2. Create a usability file containing all the usability questions to ask the subject at the end of the experiment.
3. Create a stimuli file containing html links for each of the stimuli for the experiment. Simuli need to be previously created and stored as html or gif files.
4. Run the experiment builder to create the experiment. This program needs as input the name of the experiment, a brief description of the item being designed, and the names of the questionnaire, usability, and stimuli files. Additional configuration information may also be needed so it knows where to store everything.
5. If necessary, copy the resulting html file(s) to the correct location.

• This shouldn't be too difficult. I just need to refresh my memory on how this is done!

1. Prepare e-mail messages for each subject. This message should contain the set of subproblem orderings, and any questions that still need to be asked. The subject should be asked if they have any preferences for the orderings derived. Also, if the ordering specified by this subject did not make the final list, they should be told why and given an opportunity to agree/disagree.
2. Send e-mail messages
3. Process results. Any agreement? Disagreement? Why?

• How much more data to process do I want at this point?
• How big an effect are the issues described under data analysis going to have on this stage? I may want to pick a domain where I have an expert locally available to help interpret the results.

Implementation will be a mixture of C, html, cgi, and Java. The goal is to come up with something that can be easily configured for different domains (and their stimuli). C, html, and cgi were chosen because I am familiar with them and feel comfortable completing the project in the time given. Some Java will also be used to that I can get more experience working with it. Most of the file I/O will be done in C because it's easier that way.

A big advantage of web-based interfaces is that it's really easy to build display mock-ups. I haven't done this yet though.