Cognitive Walkthrough Francesca Rizzo 24 novembre 2004
The cognitive walkthrough It is a task-based inspection method widely adopted in evaluating user interfaces It requires: A low-fi prototype of the system A description of the task the user is to perform on the system A complete, written list of the actions needed to complete the task with the given prototype. An indication of who the users are and what kind of experience and knowledge the evaluators can assume about them Given this information, the evaluators step through the action sequence to analyse the system and its usability 2
Norman s s Model of Action The theoretical background that lays behind the CW is rooted in Norman s Model of Action The model describes seven stages of user activity : Establishing the Goal Forming the Intention Specifying the Action Sequence Executing the Action Perceiving the System State Interpreting the State Evaluating the System State with respect to Goals And Intentions and three types of distances Semantic, referential and inter-referential 3
Norman s s Model of Action Some researchers have modified Norman s Model of human action making it explicit that goals can be modified during the activity Because of a failure The user does not possess the relevant knowledge to fulfill the goal The goal is not feasible Because of a continuous and dynamic process of adjustment that is derived by the interaction with the environment, depending either on the action carried out or the produced results 4
Norman s s Model of Action Mental Activity New Goal SITUATION Goal EXECUTION GULF EVALUATION GULF Scenario distance Semantic distance Intention MEANING Evaluation Semantic distance Referential distance Action specification FORM Interpretation Referential distance Execution Perception Physical Activity Input INTERFACE Output Intereferential distance 5
Direct Manipulation The relationship between the task that the user has in mind, and the way that task can be accomplished via the interface embodies the notion of Directness. Directness is indeed impression about an interface, a qualitative feeling that we are directly engaged the semantic objects of our goals and intentions. 6
Direct Manipulation Directness is inversely proportional to the amount of cognitive effort it takes to manipulate and evaluate a system and such an effort is a direct result of the Gulfs of Execution and Evaluation The gulf of execution is the thinking required to figure out how to get something done - turning the high-level intention into specific physical actions The gulf of evaluation is the thinking required to understand what is being perceived - turning the raw sensory data into an understanding of objects, properties and events. 7
Norman s s Model of Action modified Mental Activity EXECUTION GULF New Goal SITUATION Goal Scenario distance Scenario distance If the goal is wrong (or can be ameliorated), will the user understand that the intention s/he is trying to fulfil cannot be accomplished with the current state of the world (or will s/he find out alternative goals?). EVALUATION GULF Semantic distance: Will the feasible and correct action be made sufficiently Semantic evident to distance the user and do the actions match with the intention as stated by the user? Intention MEANING Evaluation Semantic distance: Will the user properly evaluate the results, (i.e. Semantic will s/he distance be able to assess if s/he got closer to her/his goal?) Referential distance: Will the user connect the correct action s description with Referential what s/he is trying distance to do? Action specification Execution FORM Interpretation Perception Referential distance: Will the user interpret the system s response Referential to the distance chosen action correctly, (i.e. will s/he know if s/he has made a right or wrong choice?) INTERFACE Physical Activity Input Output Intereferential distance: Will the user receive feedback in the same place and modality where s/he has performed her/his action? 8
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal: input a query and define search parameters Goal The system apparently supports the intention of the user to start a search. Intention MEANING Action specification Semantic distance: Q1. Is the feasible and correct action made sufficiently evident to the user and do the actions match with his/her intention? Nevertheless starting from the empty field is not the best action possible since this imply a generic query that only returns all the instances that have that string contained in their Label 1. The system does not support the user s conception of the task domain 2. It does not encode the relevant distinctions in the domain in the same way the user thinks about them 9
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : input a query and define search parameters Action specification FORM Execution Input Referential distance: Q2. Will the user connect the correct action s description with what s/he is trying to do? Closer to the input field the interface presents another button View Relations. The user does not believe this button matches his intention nevertheless he has no idea of what his function can be The distance among the meaning of the action and its physical form is pretty high and it is represented by the distance between the entry field and the search button 10
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : understand whether the query has been processed FORM Execution Perception INTERFACE Input Output Inter-Referential distance: Q3. Will the user receive feedback in the same place and modality where s/he has performed her/his action? It takes a long while before the result actually appears: the feedback of the system on the processing time is not adequate. The feedback of the system follows the modality of the language of the system and does not match a use scenario Tipo query ricevuta: INSTANCE Generic Query: Uomo_vitruviano The feedback uses a different form from the one the user used to expressed his intention. The modality is not appropriated. 11
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : interpret and evaluate search results Interpretation FORM Perception The user feels that his query has been successful Output Referential distance: Q4. Does the user interpret the system s response correctly: does s/he know if s/he has made a right or wrong choice? 12
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : evaluate search results MEANING Evaluation Interpretation The user cannot understand whether the retrieved record is an image, a book title or something else: the meaning of the result is not immediately clear Semantic distance: Q5. Does the user properly evaluate the results: is s/he able to assess if s/he got closer to her/his goal? 13
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : elaborate a new intention and strategy to query the result New Goal SITUATION Goal Scenario distance The user discovers that he has to query the result in order to see whether he has actually found the image of Leonardo's masterpiece Intention Evaluation Scenario distance: Q6. Does the user understand that the intention s/he is trying to fulfil cannot be accomplished with the current state of the world: does s/he find out alternative goals? 14
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : query the result through one of the possible options Goal Intention Action specification MEANING The feasible and correct action is not sufficiently evident: what is the difference between Search and More? Semantic distance: Q1. Is the feasible and correct action made sufficiently evident to the user and do the actions match with his/her intention? 15
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : query the result through one of the possible options Action specification Execution Input FORM Again the user cannot tell which of the two opportunities will lead him to the desired result of discovering whether the output of the system is an image of Leonardo s drawing. The user must proceed by trials and errors Referential distance: Q2. Will the user connect the correct action s description with what s/he is trying to do? 16
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : decode system feedback FORM Execution INTERFACE Perception The user clicks on More : a pop up window appears below but it is entitled as Details Input Output Inter-Referential distance: Q3. Will the user receive feedback in the same place and modality where s/he has performed her/his action? 17
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : decode system feedback FORM Interpretation Perception Output Referential distance: Q4. Does the user interpret the system s response correctly: does s/he know if s/he has made a right or wrong choice? The user understands he made the wrong choice because the result does not match his wishes 18
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : decode system feedback Goal Intention MEANING Evaluation Action specification Execution Input FORM INTERFACE Semantic distance: Interpretation Q5. Does the user properly evaluate the results: is s/he able to assess if s/he got closer to her/his goal? The user is not able to evaluate such a result: why does the option more produce a result that does not add anything to previously existing information? The user will try with the other option. 19
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : decode system feedback FORM Execution Input INTERFACE Output Perception The user clicks on Search. In this case the inter-referential distance is wider: the result appears in the search definition area, which previously seemed to be an input area and in this case appears to be also an output area Inter-Referential distance: Q3. Will the user receive feedback in the same place and modality where s/he has performed her/his action? 20
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : decode system feedback MEANING FORM Evaluation Interpretation Perception The user perceives that something has changed but he has no means to evaluate this change: he is not able to say if he got closer to his goal. Output Referential distance: Q4. Does the user interpret the system s response correctly Semantic distance: Q5. Does the user properly evaluate the results: is s/he able to assess if s/he got closer to her/his goal? 21
Goal: find an image of the Vitruvian Man drawn by Leonardo Sub-goal : figure out what goal to pursue Intention New Goal SITUATION Goal Scenario distance Scenario distance:? Evaluation Q6. Does the user understand that the intention s/he is trying to fulfil cannot be accomplished with the current state of the world: does s/he find out alternative goals? Furthermore the system does not suggest the user how to proceed, it does not support the user s conception of the task domain: Which intention should the user now formulate in order to proceed in his interaction? The user is confused and has no more strategy: he is bound to go on exploring the various possibilities since the interface does not make the correct action evident 22
MAJOR USABILITY PROBLEMS The interface does not support the user s conception of the task domain: 1. It does not encode the concepts and distinctions of the domain in the same way that the user thinks about them 2. It does not help the user in building an adequate mental model 3. It does not provide a continuously evident representation of the potential for action The distances represented by the model of human action so broad that the user is compelled to a great cognitive effort 23
MAJOR USABILITY PROBLEMS The feedback of the system is hard to interpret, it does not match the user s expectations: What is the difference between Relations and Attributes? How can a user decode CDOC-CRM relation list? What does Define your search means? input a generic query typing a string, browse retrieved results, fill in empty fields describing an object - these fields correspond to the owing Institution description system, so that they are quite hostile to an average user! 24