- Category: Annotations
- Hits: 1389
Barnum, C. (2010). Preparing for Usability Testing. In Usability testing essentials: ready, set...test (pp. 188-229). Burlington, MA: Morgan Kaufmann Publishers,
Documents for usability test:
- screening questionnaire(s)
- pre-test questionnaire
- post-task questionnaires
- post-test questionnaire
- consent/release forms
Creating the Screening Questionnaire
Devise questions identifying characteristics of user
recruit at least three (pref 4 or 5 users) for each profile
Usually focus on proficiency questions-- how often, how long, how much. Don''t ask users to ID own levels of knowledge/proficiency. Instead, open-ended questions or ranges. See samples on 190-91
budget at least 2 hours per participant (4-15 calls per user)
Recruit Directly: customer lists, potential customers, user groups/prof organizations, ads, qual friends. Telephone most time-consuming but yields most direct/fastest results. Don''t recruit people at work. Should offer some incentive.
Recruiting Indirectly Through Agencies: Market research firms tend to produce better participants. Supply your screening questionnaire. Regular use of same agency better.
Scheduling Participants: Calling works better than email, but also email a reminder. Anticipate 10% no-shows. Need back-up plan:
- Double book
- Have a backup nearby
- Schedule an extra person or two
Further Screening on Arrival: can use same questionnaire or create additional (aka pre-test) questionnaire. Need to decide what to do if user doesn''t match profile.
Avoid creating steps/instructions (focus on process); avoid using language of the product (unless you are sure that terminology is not the issue). For each scenario, create post-task questionnaire for immediate feedback.
If you want to test documentation, instruct users to work as normal and observe whether they consult documentation or direct users to documentation.
Creating Post-Task and Post-Test Questionnaires
Likert scale; or neutral statement (how difficult? very easy, somewhat easy, neither easy nor difficult...); or rate level of satisfaction; or list category/present topic.
Post-test frequently asks for qual and quant responses.
Defining Team Members'' Roles
Minimum three people:
facilitator: (direct interaction with participant) aka briefer/ debriefs.
- creates checklist of tasks/reminders
- consent/release forms (nondisclosure; cosent)
Logger: fastest typist
- observation form
- Camera/video operator: equipment
- narrator: assists logger in capturing findings
- help desk operator: person most familiar with product
- test administrator: manages all phases of projects (often combined with facilitator)
What makes a good moderator?: relaxed; practices neutral "prods"; watch vid tape of self; avoid use of loaded words (test, subjects); some positive reaction to participation; don''t talk about participants in public spaces; gracefully decline inappropriate participants; run pilot session; schedule enough time between sessions; protects privacy; has checklist; uses script; dresses sim to participants; strategies for getting feedback from shy participants; knows when to end task; may give tips on using software more effectively (at end of study).
Conducting the Walkthrough and the Pilot
Walkthrough: chance to test equipment, scenarios, etc. Recruit a tolerant user-- can deal with mistakes and confusion. Purpose is to find flaws in the plan. Tech rehearsal!
Pilot: 2-3 days before full testing begins. Chance to practice approach, etc. Pilot participant must match user profile.