Usability Testing Handbook

Handbook of Usability Testing by Jeffrey Rubin

Designers (who have usually been hired for technical expertise rather than design – communication – expertise) need to realize that they are not designing products but a relationship of product and human (reminds me of teaching!!) Designer needs to help human focus on task at hand not the means by which to do the task (this is different in subtle ways in what teaching is about)

Three principles 1) an early focus on users and tasks (as in effective speech development) 2) empirical measurement of product usage 3) iterative design whereby a product is designed, modified, and tested repeatedly

Usability goals and objectives
1) usefulness 2) effectiveness (ease of use) 3) learnability (this is a key new idea and related to teaching) – can operate the system after a predetermined amount and period of training, an infrequent users can relearn the system after a period of inactivity 4) attitude (likeability)

Four types of usability testing
1) exploratory – early in the process, seeking to answer fundamental questions and product does not have to be fully functional, measure high level
2) assessment – early to midway, after high-level design or organization has been established, seek to evaluate lower-level operations
3) validation – nearer toward release, participants get little help from test monitor
4) comparison – can be done at any of the above stages, provide alternatives, no “winner”, usually best design is a combination of alternatives, the best aspects of each design to form a hybrid

basic elements of usability testing (and this is different from classical research)
1) development of problem statements or test objectives rather than hypotheses
2) representative sample of end users
3) representation of actual work environment
4) observation of end users
5) collection of qualitative and quantitative performance and preference measures
6) recommendations of improvements

frustration is inevitable for participants, need to be careful as test monitor not to interfere with this too much, the frustration is what tells us what we need to know, be impartial, it is not the problem with the participant but with the product but participants will still blame themselves, encourage them to freely explore areas without concern for looking good

In debrief (and in any kind of assessment) good to focus on what the participants expected to happen, begin by letting the participant say whatever is on their mind

Before giving the test, give it to yourself, in giving the test to others expect the unexpected

Before starting, ask the participant to parrot back what they have been told about the test and process

Provide prerequisite training if necessary, remember posttest questionnaires (this can focus more on preference while the test focuses on performance)

Can test two participants at once (use their conversation like a “think outloud”), (this could work for testing student powerpoints)

Book to check out The Design of Everyday Things by Donald Norman, and newsletter called Common Ground (from Usability Professional Association)

New Horizons – usability for not just product development but for also assessing the entire product ownership life cycle, some automated usability evaluations are being directly built into software products (reminds me of remote car article in Wired, a step toward artificial intelligence), codewriting will also become more automated which may mean a greater need for usability testing

Don't Think of an Elephant

Don’t Think of an Elephant by George Lakoff (2004)
www.rockridgeinstitute.org

Frames are mental structures that shape the way we see the world – our goals, plans, acts, and what counts as good or bad outcome of actions

Cognitive unconscious – structures in the brain we cannot consciously access but know of their consequences: our reasoning and what counts as common sense

Language also tells us, all words are defined relative to conceptual frames, hearing a word activates its frame in our brain

Reframing is changing how someone sees the world and common sense, we need new language for new frames

Exercise – Don’t think of an elephant! Whatever you do, do not think of an elephant. (no one can do this, elephant evokes a frame, when we negate a frame we evoke it) (Nixon - I am not a crook)

A basic principle of framing – if you are arguing against the other side don’t use their language (this language picks the frame, one you don’t want)

Framing is about getting language to fit your worldview, but ideas are primary – the language carries the ideas (this is about metaphors)

Metaphor for nation as a family – the Founding Fathers, the Daughters of the American Revolution, sending our “sons” to war, a natural metaphor to compare large social groups such as a nation in terms of a small one, like family

Our nation can be compared to strict father nation (Republicans) or nurturing parent family (Democrats), we have both models imprinted on our synapses as passively and actively, we can use them both but in different parts of our lives

Strict father approach = protect family in dangerous world, support family in difficult world, teach children right from wrong – pursue self interest and grow up to be self-reliant is moral, good people will be (deservedly) rewarded, the world is dangerous and difficult, children are born bad and must be made good, need to be disciplined

Other metaphor for foreign policy = rational actor and nation as a person (“friendly” nation, “rogue” nation, “enemy” nation, “adult” nation, “child” –developing- nation), maximize self interest is the rational thing to do

Nurturant parent metaphor – empathy (caring and feeling how others feel) and responsibility (taking care of others whom we feel responsible), the world is basically good and can be made better – seek fulfillment in life, fairness, freedom, open two-way communication

Each of these frames forces a certain logic, for facts to be accepted they must fit the frame, if the facts do not fit the frame, the frame stays and the facts bounce off (myth= the truth will set us free, tell the facts and since people are basically rational they will reach the right conclusions)

Neuroscience says that our long-term concepts that structure how we think are “instantiated” in the synapse of our brains. Concepts are not things that can be changed by telling us a fact. It not, it is not heard or mystifies us – seen as irrational, crazy or stupid.

Economics – people are rational actors following their self-interest, this does not bear out to true in how people really think, people vote their identity (who they identify with) and their values not their self-interest, and it’s not as if they don’t care about self-interest but it isn’t as simply that

We tend to have both family metaphors – the john wayne movie and the cosby show, both worldviews are widely present in our culture, people do not necessarily live by one worldview all the time

Orwellian language – language that means the opposite of what you say, used by conservatives when they are weak, when they cannot come right out and say what they mean, Orwellian language points to weakness, note where this is for this is were they are vulnerable

Language can be used honestly or harmfully, the right language starts with ideas, ideas come first and are used to frame

When you think you lack words you really lack ideas, ideas come in the form of frames, when frames are there the words come readily

Hypocognition – lack of ideas that you need, the lack of a relatively fixed frame that can be evoked with a word or two
Study by bob levy in Tahiti – many suicides, Tahitians feel grief but have no concept of it or name for it, no rituals or seen as a normal emotion and end up committing suicide too often

Hierarchy of values (what’s the top value) and strategic initiatives (has multiple, long-range effects) instead of issue by issue approach, also slippery slope strategic initiatives (taking small steps toward a larger issue, understand the consequences of steps)

In reframing be proactive, not reactive, practice every day on every issue, use your frames not theirs, use then because they fit the values you believe in

Even if frame doesn’t fit the facts, there is grain of truth to it and that holds

Campaigning as marketing metaphor – look to a list that most people support, move to the “right” on some issues to get the centrist vote, a list is not a moral vision, does not present voters with an identity (someone to identify with)

Marriage – a word, and a metaphor – symbolizes an institution, life goal, dreams, dates, gossip, anxiety, shower, wedding, rituals, invitations, vows, honeymoon, family, children, grandchildren, in-laws, games, graduations, a journey, partnership, complementary parts, sacrament, home, social status, sex

Mirror neurons – these fire either when we perform an action or we see someone perform the same action. And these are connected to the emotional parts of our brain. The basis of empathy

Gandhi – Be the change you want

Metaphors can kill (http://philosophy.uoregon.edu/metaphor/lakoff-1.htm) war is fought against a person (Saddam Hussein) not the Iraqi people, protesters are against the country (not patriotic) not Bush
Nation as a person – a “just” war, self-defense story or rescue story, in both a hero, a crime, a victim, a villain (in self-defense story victim and hero are the same), villain can’t be reasoned with and needs to be defeated or killed, victim is innocent and beyond reproach

Lies – what is difference between lies and exaggerations, misleading statements, mistakes, rhetorical excesses
For a linguist when considering whether statement is a lie the least important consideration for most people is whether it is true
More important is – did he believe it, did he intend to deceive, was he trying to gain an advantage or to harm someone, is this a serious matter or trivial one?
Most people see that if the statement is false but if he believed it, wasn’t trying to deceive, was not trying to gain advantage then no lie
If in the service of a good cause then it is a white lie, based on faulty information then an honest mistake, used for emphasis then exaggeration
Difference between a lie and betrayal of trust

What divides progressives (understanding change)
Local interests
Idealism vs. pragmatism
Radical change vs. moderate change
Militant vs. moderate advocacy
Types of thought processes (socioeconomic, identity politics, environmentalism, civil libertarianism, spiritual and antiauthoritarian)

What can unify (understanding change)
Values coming out of a basic vision
Principles to realize vision
Policy directions that fit values and principles
Brief ten-word philosophy that encapsulates all of the above

How is framing different from spin, manipulation, and propaganda? Framing is normal, every sentence is framed in some way, when we say something we believe then we are using frames that we think are relatively accurate, spin is a manipulative use of framing, it is used when something embarrassing has happened or said and is an attempt to put a clean frame around it and make it look normal or good, propaganda is an attempt to get the public to adopt a frame that is not true and is known not to be true for the purpose of gaining control or power
Avoid deceptive framing, speak from what you believe

Once your frame is accepted in the discourse, everything you say is just common sense
Guidelines
Show respect
Respond by reframing
Think and talk at the level of values
Say what you believe

Access by Design

Access by Design by Sarah Horton (2006)

Introduction by Ben Shneiderman
Three waves of web dissemination 1) users learn about possibilities and get connect, one in six 2) second wave users create web pages, produce content and disseminate ideas, music, photos etc., shift from information access to content generation = ebay, wikis, blogs 3) transformation from information to action, effective international development and innovation education and safe neighborhoods and health care and environmental protection and conflict resolution = requires open discussion and broad participation

Preface
Web has democratized design – both process and attendant responsibilities
Best practices – depends on our definition of users
Stages of web development 1) graphic design 2) user-centered design 3) web accessibility 4) web standards 5) universal design 6) universal usability
Layers of web design (a) function (b) interface (c) content which is the ends if function and interface are the means (Horton focuses on function)
Design and designers – we don’t think of ourselves as designers but we make soup, sand castles, we are not making t-shirts, one size can fit all, the web makes it so

Intro
Designing tools to fit the task so that it becomes part of the task (sleeping in bed, getting time from clock)
*Form follows function (architect Louis Sullivan) observed in nature, this relates to communication being audience driven
Divides visual users and nonvisual users – critical differences and ways of using the web, nonvisual users include text readers and search engines (machine use of web site)
Two basic function of web site 1) communication of information, key is accessibility 2) user interaction, key is functionality
Web is universal because powered by text, test can be read and understood by computers
1) when web content is presented as text then communication
2) when web content is structured then meaningful communication can occur
3) when web pages are operable and functional according to expectations, interaction can occur
4) if web pages are flexible and device-independent, communication and interaction can occur for more users (harder for physical world to be customized, most is fixed in form, with one-size-fits-all someone has to compromise)
Conclusion – designers must compromise, quality site not simple to design but can be as hard as building a site that is not good, need to understand and honor the web, also partnering with users, both are responsibly and collaborative
Feels counterintuitive that users can change elements of web site and violate design conventions, but we cannot make design decisions that work for everybody so users must be able to make their own decisions
Applying conventions undermines the strengths of the web
Design collaboration with users requires two things 1) design for transformation = flexible elements and overall design must hold up to change, keep setting fluid so that settings can be overrode without breaking the page 2) recognize and respect boundaries of user domain, the web gives designer control to users

Chap 1 fundamentals
Design simply “typography exist to honor content”, well designed web site has enough emphasis to spark interest and draw attention to important elements but not to enough to distract or limit functionality
Build fallbacks is one UD strategy, html is generally the most universal format and can be a viable alternative format to address accessibility, html is the web’s native language and web access software such as browsers and search engines are written to read it (not the most powerful of tools but none others are explicated designed to provide access)
More fallback – video might fall back to slide show, then audio, then image and finally text
From mark-up perspective three attributes need to be accounted for 1) keyboard accessibility 2) flexibility 3) user control
Keyboard control means using tab or arrow key to select elements and return or enter key to activate them (two steps), language counts so should be able to understand purpose of element without expanding focus to surrounding context
Web pages are inherently dynamic, design used to be for static items such as posters, books or billboards with only one view in mind, use relative measurements (percentages)
The potential for user customization offers an opportunity to redefine the relationship between designer and user, the boundary between the domain of the designer and the user

Chap.2
Device-independent format is dependent upon separating content and presentation
Creating html documents is not a visual process but an intellectual one, use html for content and use CSS for presentation
Use html mark-up tags to create meaning (structural elements)
CSS allows for reduced web pages cluttered with presentation markup, allow for audible reading of html pages better, can use one master sheet and effect changes across the site more easily and consistently
But pages must be able to function with style (sheets)
Linked (external) style sheets are better than embedded (or inline) ones

Chap. 3 text
Text can be sized, colored, style, copied, pasted, indexed, searched – images and graphics text cannot
Use markup tags to indicate information hierarchy and use flexible widths
Avoid using color alone to convey information

Chap. 4 images
Do not have to abandon images – this would make it difficult for those who depend on images for help
Images are not for “spicing up” or because it is a visual medium, but for providing information, establishing context, providing direction, establishing brand or identity, but images draw the eye first and must be used purposely

Chap. 5 data tables Chap. 6 layout tables
Software cannot distinguish “real” tables from layout tables
Layout is best handled by css, use tables for layout only when necessary
One benefit of structured documents is their ability to be indexed and then discovered by search engine software
Software that reads web pages linearizes tables
For both visual and nonvisual users we need to design code as well as visual display

Chap. 7 frames
Avoid using frames

Chap. 8 lists
Markup lists so that lists (including navigation bars, tabs and breadcrumbs) can be understood by nonvisual users

Chap. 9 forms
Forms should not be sole means of communication with site owners, have alternative methods (email, phone #, snail)
After links, forms are the controls we use most frequently to interact with the web
Importance of labeling
Form design is a process of simplification and clarification, keyboard accessibility needs to be made
Information flow must be predictable, anytime a user can successfully predict what come next, usability is enhanced

Chap. 10 links
We spend much of our time on the web following links, links must be displayed as text
Use descriptive text for links rather than “click here” etc. (doesn’t work for skimming page, will need to expand focus to surrounding context)
First word is important, one most likely to grab user’s attention
Link underlining is a user-defined setting
Differentiate between visited and unvisited links
Use “you are here” orientation

Chap. 11 color
Check out Colour Contrast Analyzer
Do not use color alone
Important to have color differences in brightness and hues (purple and yellow)

Chap. 12 audio and video
Provide alternative formats for media-based content (text)

Chap. 13 interactivity
Web is a client-server application, this allows for navigation to find information but is less functional as a software application, the web does not support page-level interactivity as we have come to expect from desktop and cd-rom applications, links and forms are the only native modes of interactivity

Chap. 14 editorial style
Need for accessible language
Need to be structured for skimming = information broken into segments, and using headings, lists
Skimming is enhanced with initial key words

Chap. 15 page layout
The ultimate test of a web page is how well it performs when read by software
Begin with primary page content as opposed to advertising and navigation links, put content as close as possible to the beginning
Feature creep – extensive navigation options, global navigation, local navigation, page navigation, bread crumbs, link creep
Be context sensitive for navigation, provide appropriate links based on content and available tasks rather than extensive link lists covering every possibility, some backtracking is reasonable
Design is a moving target, designer can only “suggest” attributes