- Category: Annotations
- Hits: 1988
Takes issue that many traditional usability tests "focus on function...and not on the process of how the information is used" (3). Argues divorced from reality. Uses as example that time on task for single actions can hide "turmoil beneath" (3). Notes how multiple audiences (needs/presentation formats) further complicate; explains that writing for the LCD can impair comprehension in other groups (3).
If focus on usability of individual components, easy to measure-- but often privileges one aspect "without having a solid theoretical foundation on which to base the priveleging" (5).
A fundamental problem we must address is whether Mirel is correct in claiming that the usability of complex design problems is more than the sum of the individual components (5). "Complex information is multidimensional. There are no simple answers, there is no single answer, and there is a dynamic set of relationships that change with time and in response to situational changes" (5).
The question becomes, then, how do we test it? First, change view that usability is a sum of the parts (6). Any "meaningful" usability measurement needs to "capture, or at least account for, those complex interactions" (6). Creating a complete interaction tree is impossible-- can't be thoroughly tested. "The standard guidance is to pick and choose the features to test in order to focus on the areas that must have good usability" (6).
"Do the methods capture the usability within the complexity of actual system use, or have they overly simplified (for a trivial example, consider simply minimizing clicks) or ignored aspects of the situation (for example, dealing with partial doses given two hours apart) that render any usability results suspect?" (7)
Examining the Definition of Usability
ISO: "effectiveness, efficiency, and satisfaction" (8)
Quenesbery (2003): 5Es: effective, efficient, engaging, error tolerant, easy to learn. Creates spoked wheel; each E may be different size depending upon application (e.g., easy to learn may not be as important if system constantly being used by user). Es may be in conflict (efficient data entry may not lead to effective patient care).
Defining relative size of 5Es requires considering both situation and audience.
Albers (2004) defines situation: "the current world state which the user needs to understand. An underlying assumption is that the user needs to interact with a system to gain the necessary information in order to understand the situation (p.11)" (10). Current world situation not static, have error/undesirable conditions, memories may complicate.
Audiences: "include people with a range of knowledge about the topic, the system, and the frequency of use" (10). If group has high/middle/low for each dimension, 9 component subgroups; with h/m/l for each subgroup, 27 components = single group.
Final complication: not just button pushing. Information use. Layered model: conceptual (main concepts), semantic (functionality of system) syntactic (interaction tokens to create semantics), lexical (structure of the tokens), pragmatic (how to generate them physically by user actions and i/o elements) (11).
Syntactic, Lexiconal, and Pragmatic (How-To Layers)
make up the lowest levels and direct interface manipulation; traditional usability testing explores almost exclusively these three layers (11). If person has problem with data manipulation-- perform usability testing that reflects these three. Testing these levels do not ensure usability, but they cannot be ignored-- contribute to look and feel of system, resulting user satisfaction and perceived user-friendliness (11). Notes that perceived user-friendliness tends to result in better decisions (12).
Semantic (Functionality Layer)
semantic represents sequences of user actions and system functions (12). Typically no "correct" path, so no way to test as good/bad passed/failed. Must analyze interaction as a whole. Need to consider how creative people may get working in the system (e.g., hacking, so to speak). Traditional usability can end when person finds information, but with complex, goal is using the info. But how do people respond to large amounts of info? "Data availability paradox": people tend to ignore it (12). Asks if complex information usability about data overload and ways to avoid it?
conceptual connects the mental model of the users to the main concepts of the system (13). A mental model (cognitive model, cognitive schma, mental schema, scripts) corresponds to cognitive layout person uses to organize info in memory (13). Enables making connections among disparate bits of info (Redish, 1994) (13). Unlike with simple information, no 1:1 mapping; testing needs to change.
For simple system, 1st step is to define user's needs. For complex, "first step is to define the communication situation in terms of the necessary information processes and mental models of the users" (users need contextual awareness) (13).
Significant factor in usability testing of complex information is to "ensure that the proper mental model gets activated and that the presentation supports easy population of that model" (14). -- once activated, people don't swith mental models.
"The errors occur when the proper information has been correctly perceived, but the relative importance of these information elements has been misinterpreted, often because the information did not receive proper salience for the person's current goals" (14). Likens it to tunnel vision, info disregarded-- usability test must consider and identify places where they may affect a person's interaction with the information. (14)
not a simple-information/easy-answer mentality. "For usability test results to reflect the real-world operation of a system, the complexities of the design, the information, and the human-information interaction must all be considered and made part of a usability test's evaluation, data analysis, and design change recommendations" (14). Complex usability testing must be ITERATIVE as well.