PROJECT 1 - Requirements, Heuristics, Qs & Gestures

Due:

Project 1 is due in class on M 20 Mar

Goals:

The goals of this project are to provide you with an overview of the field, by asking you:

1. To practice reverse-engineering of an interface design, in order to infer from the design what the requirements were;
2. To expose you to some detailed and seminal research in HCI and allow you to evaluate and respond to it;
3. To provide you with an opportunity to do a small usability study to evaluate an interface;
4. To expose you to some current work on interfaces to allow you to evaluate and respond to it.

Rationale: HCI professionals are required to know about interface design requirements. They should be able to discover, evaluate and use relevant HCI research, and be able to keep current with that work. They should also be able to evaluate and critique other interfaces, as well as their own, as part of the design process. They should be able to learn from interface users about the quality of interfaces. In addition, an HCI professional must know what new devices and techniques are being proposed in the field. All these skills and this knowledge allows them to design and produce good interfaces that demonstrate HCI principles.

Part 1:

1.1 Task:

The task is to carefully study this old WPI home page, find all the design decisions that have been made, and try to infer why all those decisions were made that way. This leads you to infer the Requirements from the Design: the opposite of the normal design process. What you are doing is called "Reverse Engineering", as engineering usually proceeds from requirements to design.

Hints: When the web site was developed, they needed to create information that would help them design the site. This includes information about 'audience', 'purpose', 'objectives', and 'domain'. That's one way to view the web page in order to make inferences about the requirements. It is also useful to reverse-engineer the design by considering the syntax and semantics of every item, as well as item grouping on the page, and the relationships between groups. You already saw all these ideas in the first class.

1.2 Hand In:

Hand in a clear, structured (i.e., not an essay), printed (i.e., on paper) explanation of what you think the Requirements were for the old WPI web page. Explain why they were chosen as the Requirements, and what evidence made you think that. Also describe all of the main interface design decisions. Note that we are not asking you for any sort of evaluation. Part 1 can be up to 5 printed pages.

Part 2:

2.1 Task:

Some published articles evaluate or record successful use of HCI techniques on real products: in this case for user interface evaluation. The paper you will study is a classic cost-benefit analysis of three usability analysis techniques. It refers to seminal HCI research that you should know about.

{Note: Here are some notes about GOMS; here are some heuristics (definition) for the Heuristic Evaluation method}.

Your task is to read:

J. Nielsen and V. Phillips, Estimating the Relative Usability of Two Interfaces: Heuristic, Formal, and Empirical Methods Compared, Proceedings of Human Factors in Computing Systems, ACM INTERCHI'93, ACM Press, April 1993, pp. 214-221.
(also at web.cs.wpi.edu/~dcb/courses/CS3041/p214-nielsen.pdf)

2.2 Hand in:

Hand in printed answers to all of the following questions. Keep answers precise, clear, and brief. Make sure you explain, as just answering "yes" or "no" isn't adequate. What you hand in for this part (part 2) of project 1 should be no more than 2 printed pages.

1. What is the goal of the research described in this paper?
2. What are two advantages of GOMS analyses?
3. What is the main reason for using a Heuristic method?
4. What is a "cost-benefit" analysis?
5. Under what conditions were Heuristic Estimates better?
6. Do they suggest relying on "relative usability estimates" or "absolute usability estimates"?
7. What is meant by a "cold estimate"?

Part 3:

3.1 Task:

Your task is to do a small, informal usability study with two users. Observe their actions and use their feedback to provide a critical evaluation of their experiences using the current WPI web interface. Make sure you know the interface well yourself before starting this study!

Select your users to gain maximum usability information. Try to select people who have little or no experience with the WPI interface. Study each user in a separate session, treating them in the same way as much as possible. Although this is difficult, prefer people who are not in, or have not taken, this class. Please do not use subjects who have already done this task for another classmate.

The goal of the experiment is to discover whether the answers to some basic questions are easy to find.

The three questions you must use are:

• Which WPI Project Centers are in Asia?
• Is there a Ruth Brown at WPI?
• Are freshmen required to buy a computer before coming to WPI?

Work with one subject at a time. Read them the instructions first. You will ask each subject those three questions. Read them a question, and give it to them on paper, then let them work to find the answer at the web site for no more than four minutes before stopping them. Start each subject at the WPI Home page for each question. Repeat the process until all questions have been used. It is possible that they may not find answers to the questions.

The instructions that you should read them, and provide to them on paper, are given directly below.

Instructions: I will ask you three questions. After each question you will have up to 4 minutes to find the answer starting at the WPI home page. You may use the search tool if you want. Make comments out loud as you use the interface about what you are doing and why, and what you feel about the interface. When are you surprised? Unsure? Lost? Happy? Frustrated? If you find the answer, point it out to me immediately: we'll then return to the WPI main web page before the next question. I can help you with that if necessary. Once you are finished you will be asked a few questions that will help us clarify what happened during this activity.

You must not guide them or answer any questions about the web page itself. Make notes along the way about what they did and said. Afterwards, ask them the usability questions given below.

The usability questions you ask each of them are:

1. How easy on a scale from 1-5 (easy to hard) was it to find each of the answers?
2. How much did searching for an answer help you?
3. How could the search tool be improved?
4. How could the WPI web pages be improved?

3.2 Hand in:

Summarize the results of the usability study by handing in printed answers to all of the following questions. Keep answers precise, clear, and brief. What you hand in for this part (part 3) should be no more than 4 printed pages.

1. For each of the three questions:
   • Did they find the answer in less than 4 minutes?
   • Did they search?
   • Was searching the first thing they tried?
   • How many links did they select (i.e., how many "clicks")?
2. Based on the subjects' feedback, what was the most confusing aspect of the way that the interface looked or behaved?
3. Based on their behavior, when they looked at a link could they always anticipate what information was going to be displayed when they selected the link? Where was that most untrue?
4. Based on their ratings, how easy (or hard) was it for them to find the answers?
5. Based on their suggestions, what changes might be made to improve searching?
6. Based on their suggestions, what changes might be made to improve the WPI web pages?

Part 4:

4.1 Task:

There is a lot of current research on new techniques for interaction. Many are introduced without being fully evaluated, and many have their evaluations kept company confidential. The issues include physical and cognitive load/difficulty, including time to complete a task, error rate, and mental or physical stress.

Your task is to read the paper:

S. Zhai & P. O. Kristensson, The word-gesture keyboard: reimagining keyboard interaction, Communications of the ACM, 55(9), September 2012.

4.2 Hand in:

Hand in a printed version of:

1. your view of the most important information in the paper (1 page);
2. your opinions about the strengths, weaknesses and utility of some of the techniques it mentions (1 page);
3. your thoughts about the methods used for evaluation of the techniques (0.5 page);
4. your idea for an important new application for this kind of interface (0.5 page).