Usability Testing Handbook

Handbook of Usability Testing by Jeffrey Rubin

Designers (who have usually been hired for technical expertise rather than design – communication – expertise) need to realize that they are not designing products but a relationship of product and human (reminds me of teaching!!) Designer needs to help human focus on task at hand not the means by which to do the task (this is different in subtle ways in what teaching is about)

Three principles 1) an early focus on users and tasks (as in effective speech development) 2) empirical measurement of product usage 3) iterative design whereby a product is designed, modified, and tested repeatedly

Usability goals and objectives
1) usefulness 2) effectiveness (ease of use) 3) learnability (this is a key new idea and related to teaching) – can operate the system after a predetermined amount and period of training, an infrequent users can relearn the system after a period of inactivity 4) attitude (likeability)

Four types of usability testing
1) exploratory – early in the process, seeking to answer fundamental questions and product does not have to be fully functional, measure high level
2) assessment – early to midway, after high-level design or organization has been established, seek to evaluate lower-level operations
3) validation – nearer toward release, participants get little help from test monitor
4) comparison – can be done at any of the above stages, provide alternatives, no “winner”, usually best design is a combination of alternatives, the best aspects of each design to form a hybrid

basic elements of usability testing (and this is different from classical research)
1) development of problem statements or test objectives rather than hypotheses
2) representative sample of end users
3) representation of actual work environment
4) observation of end users
5) collection of qualitative and quantitative performance and preference measures
6) recommendations of improvements

frustration is inevitable for participants, need to be careful as test monitor not to interfere with this too much, the frustration is what tells us what we need to know, be impartial, it is not the problem with the participant but with the product but participants will still blame themselves, encourage them to freely explore areas without concern for looking good

In debrief (and in any kind of assessment) good to focus on what the participants expected to happen, begin by letting the participant say whatever is on their mind

Before giving the test, give it to yourself, in giving the test to others expect the unexpected

Before starting, ask the participant to parrot back what they have been told about the test and process

Provide prerequisite training if necessary, remember posttest questionnaires (this can focus more on preference while the test focuses on performance)

Can test two participants at once (use their conversation like a “think outloud”), (this could work for testing student powerpoints)

Book to check out The Design of Everyday Things by Donald Norman, and newsletter called Common Ground (from Usability Professional Association)

New Horizons – usability for not just product development but for also assessing the entire product ownership life cycle, some automated usability evaluations are being directly built into software products (reminds me of remote car article in Wired, a step toward artificial intelligence), codewriting will also become more automated which may mean a greater need for usability testing