6.893 User Interface Design & Implementation
Fall 2003

GR5: User Testing and Final Report

Due Wed, Dec 10, in class

In this group assignment, you will evaluate your interface with a small user test, iteratively improve it based on the results of the test, and write a final report.

User Testing

Find 3 representative users from the general MIT community (students, staff, faculty). None of your users should be enrolled in 6.893. All your users should be members of your target population. All should be willing to participate voluntarily.

Prepare an informed consent form for your users to sign, based on the COUHES template (Microsoft Word).

Prepare a briefing and three tasks. These may be the same ones that you used in paper prototyping, but you may want to improve them based on feedback from the paper prototyping.

You may, if you wish, also prepare a short demo of your interface that you can use to show your users the purpose of the system. The demo should be scripted, so that you do and say the same things for each user. It should use a concrete example task, but the example task should be sufficiently different from the test tasks to avoid bias. The demo option is offered because some interfaces are learned primarily by watching someone else use the interface. Think carefully about whether your interface is in this category before you decide to use a demo, because the demo will cost you information. Once you've demonstrated how to use a feature, you forfeit the chance to observe how the user would have used it otherwise.

Pilot test your briefing, demo, and tasks, before you actually bring in any users. Use another group member or another member of the class for your pilot testing.

Conduct a formative evaluation with each user:

Obtain informed consent.
Provide your briefing and (optionally) demo.
Then provide the tasks one at a time, observe, and take notes.

One member of your group should be the facilitator of the test, and the rest observers. Single-person groups must both facilitate and observe. Watch and record critical incidents.

We don't recommend that you videotape your users. However, if you want a record of the user test to supplement your notes, you may try using screen capture software, such as Camtasia Studio.

Redesign

Collect the usability problems found by your user tests into a list. Assign each problem a severity rating (cosmetic, minor, major, catastrophic), and brainstorm possible solutions for the problems.

Then, fix your implementation to solve as many problems as you can, giving priority to severe problems.

What to Hand In

On the last day of class, Wednesday, December 10, you should hand in a final hardcopy report describing your interface. The report should have the following parts:

Problem (1-2 pages). What user problem are you trying to solve? Who are the users? What are their tasks?
Design (4-5 pages). Describe the final design of your interface, including any redesign you did after user testing. Illustrate with screenshots. Point out important design decisions and discuss the design alternatives that you considered. Particularly, discuss design decisions that were motivated by the three evaluations you did (paper prototyping, heuristic evaluation, or user testing).
Implementation (1-2 pages). Describe the internals of your implementation, but keep the discussion on a high level. Discuss important design decisions you made in the implementation. Also discuss how implementation problems may have affected the usability of your interface.
Evaluation (1-2 pages). Describe how you conducted your user test. Describe how you found your users and how representative they are of your target user population. Describe how users were briefed and what tasks they performed. Discuss the critical incidents you observed. Discuss any remaining usability problems that you didn't solve in your final design, and suggest solutions.
Reflection (1 page). Discuss what you learned over the course of the iterative design process. If you did it again, what would you have done differently? Focus in this part not on the specific design decisions of your project (which you already discussed in the Design section), but instead on the meta-level decisions about your design process: what features to prototype, what prototype techniques to use, and how to evaluate the results.