Integrating User Evaluation into Software Development Environments

Transcription

1 Integrating User Evaluation into Software Development Environments Yael Dubinsky, Tiziana Catarci, Shah Rukh Humayoun, and Stephen Kimani Dipartimento di Informatica e Sistemistica Università di Roma "La Sapienza" Via Ariosto - 25, 00185, Roma, Italy {dubinsky, catarci, humayoun, kimani}@dis.uniroma1.it Abstract User-centered design (UCD) guides user evaluation and the design of the user interfaces (UI) as part of the software development process. In this paper, we present our work to integrate UCD activities into software development environments. Specifically, we present an Eclipse plug-in that is under development that its main capability is to manage the UCD activities within the development environment thus providing Eclipse perspectives for the evaluation manger role and for the UI designer role enabling defining and executing UCD experiments, viewing experiments results, and manipulating the implied UI design tasks. We manage six teams of six students in each team who work to develop the plug-in. We report on the first iteration of development including the requirements and parts of the high level design. This work emerges from the DELOS tasks that relate to user evaluation techniques. Categories and Subject Descriptors H5.2. User Interfaces: User-centered design, K.6.3 Software Management: Software development, Software process. General Terms Management, Measurement, Design Keywords User-centered design (UCD), agile software development, Eclipse plug-in 1 Introduction The user-centered design (UCD) approach is used to develop the systems by positioning the real users at the centre of designing activities such as: by representing or modeling users in some way like scenarios and personas; through user testing of prototypes; by involving the users in making design decisions (e.g. through participatory design). The approach s focus is on the ease of usability for users by involving them in designing and development activities. In many cases customers can benefit from this approach especially when they cannot represent real users. By involving the real users in designing and development of systems, we reduce the risks of product failure and the cost in the long run. Further, we increase the product quality. Variations in activities arise in different UCD methods [Dix et al., 2003; Sharp, Rogers and Preece, 2007], and still the Human-Computer Interaction (HCI) community lacks to agree upon a precise definition of UCD method or process [Blomkvist, 2006, Gulliksen et al., 2003]. However, in [Gulliksen et al., 2003] there is a set definition of twelve principles for designing and developing systems with focus on UCD and a process as: User-centered systems design (UCSD) is a process focusing on usability throughout the entire development process and further throughout the system life cycle (p. 401) The International Organization for Standardization (ISO) has also defined the standard

2 guidelines to deal with different aspects of HCI and UCD; in particular, ISO/DIS provides the guidance on user-oriented design process. Other relevant ISO standard guidance are ISO , ISO TR A detailed discussion about the methods, processes, guidelines and prototyping activities in UCD can be found in ISO standards 4 and in [Dix et al., 2003; Sharp, Rogers and Preece, 2007]. We identify a lack of UCD management which we define for a specific software project as the ability to steer and control the UCD activities within the development environment of this project. This is based on our observation that usually users are not involved in the process of development and if involved there is no implicitly impact in the Integrated Development Environment (IDE) that is used. We formulate an Eclipse plug-in to support UCD management in software projects that work according to the agile software development approach 5 which becomes mainstream in the last decade. One expression of integrating UCD management in agile software projects [Blomkvist, 2005; Detweiler, 2007; Hudson, 2003; Hwong et al., 2004; McInerney and Maurer, 2005] is that in every iteration planning there are UCD tasks in additional to the development tasks; the results of these tasks are presented in the iteration presentation. Another example is the role of the design evaluator whose responsibilities are to plan the evaluation of the user interfaces design, gathering and analyzing evaluation data, and recommending UCD tasks for next iterations accordingly. Our aim is to define a framework by integrating UCD activities into the software development lifecycle. In this work, we present a plug-in tool that aim at getting complete benefits from such a framework. The remainder of this paper is as follows. In Section 2 we describe the user s role in software development processes as well as specific use cases from which the suggested plug-in is emerged. In Section 3 we present the requirements of the first iteration of development and some parts of the high level design. In Section 4 we conclude. 2 The Users Role in Software Development Processes The standard ISO 9241 identifies the following as the most useful indicators for measuring the level of usability of a product: Effectiveness in use, which encompasses accuracy and completeness through which users achieve certain results. Efficiency in use, which has to do with the resources utilized in relation to accuracy and completeness. Satisfaction in use, which includes freedom from inconveniences and positive attitude toward the use of a product. In light of this standard, we bring the perspective of the customer and the user for whom the software is developed. We distinguish between the roles of the customers and the users while focus on the way users should be involved in software projects. The customer role in software development environments is central and is based on on-going communication between the customer and the team members, both with respect to the requirements and to the way testing and checking of the developed product are performed. This communication is established with the aid of several practices, one of which is the planning session. During this session, the customer observes the current developed artifacts, gives feedback, and prioritizes the work for the next iteration. The users themselves and the design that follows user evaluation are somehow neglected in software projects. A common misconception is that the customer represents all users. 1 ISO/DIS 13407: Human Centered Design for Interactive Systems 2 ISO : Ergonomic requirements for office work with visual display terminals (VDTs) 3 ISO TR 16982: Ergonomics of human-system interaction - Human-cantered lifecycle process descriptions 4 See 5 See the agile manifesto in

3 Given that the customer is one or few people who sometimes pay for the software development or have other kinds of interest with the development, the users are the major group of individuals in the context of most software projects. The UCD approach puts the users in the center of every interaction, and includes methods to deal with users evaluations and its implications to design and development. Integrating UCD into software projects can be expressed as part of the following categories: Iterative design activities - The design is updated regularly as the product evolved. The user evaluation is fostered by performing UCD tasks in each iteration of two to four weeks, and the design is updated according to the evaluation on-going outcomes. Measures Taking measurements is a basic activity in software development processes. The set of user evaluation tools is built and refined during the process and is used iteratively as a complement to the process and product measures. Roles Different roles are defined to support software development environments. The UCD roles, like for example the Evaluation Manager role are defined. In the process of implementing software projects in the academia that involve UCD activities, we have gathered data that concerns with the tool that is needed to manage the development environment. The categories that are raised are formulated as use cases to be developed as part of the tool that we present in this paper. The phrase User Perspective is used to refer to the perspective that supports UCD management. The use cases are scenarios that are taken from the world of agile teams. 2.1 Users Involvement There is a need to involve the users in the process of development. Following are two examples for use cases that relate to this category: One of the tasks during the first planning session is as follows: Explore who are the kinds of users who should use the product that we develop; what are their characteristics; what are their needs; what are their expectations from the product. The customer explains that this is an important task since he cannot represent all users and actually he does not know for sure what their exact needs are (though he is sure they will like it a lot). One of the teammates asks to be assigned to this task and estimates it as 10 hours of work for this iteration. Presenting her results after two weeks, she opens her development environment browsing over the User Perspective and shows the list of 20 users she talked with (names, titles, contact details), main issues that were learned, and one new task that has emerged for future iterations: Prepare and run a questionnaire that will enable us to extract users needs. The customer sets high priority for this new task. The project manager reviews the subjects for the coming reflection session, and sees that one of the subjects is ways to assess the usability of our product. She then sends invitations to seven users from the two different kinds of users to join this meeting. During the reflection session, one decision is made that two users will participate in each iteration planning session and their responsibility will be to give feedbacks on what presented. In addition they will help in defining three measures that will be automated thus enable teammates to receive an immediate feedback during development. 2.2 Evaluation There is a need to perform user evaluation and to manage it along the process of development. Following are two examples for use cases that relate to this category: The team leader browses over the details of the user experiment that is planned for tomorrow. He sees the number of users that will arrive, the names, and responsibilities of the two teammates that will take care of this experiment. He checks the variables that were set and the experiment flow. One of the teammates sees that the User Perspective flushes meaning new data has arrived. He clicks on it and sees that the results of the user experiment that was conducted yesterday are in.

4 He is surprised to find a new problem with high severity ranking. Examining results from previous experiments, he observes that this is a new problem and adds a note about it in the discussion area. During the next iteration planning, the experiment results are presented and among others, a measure is presented that shows two problems that are emerged by users one in normal severity and one in high severity. 2.3 Design Improvement There is a need to improve the design of the user interfaces based on the evaluation results. Following are two examples for use cases that relate to this category: The designer of the user interface views the latest design diagrams and tries different changes that adhere to the new task in this iteration. The task was added due to the last problem that was found by users. Thinking of different options, she talks with two users and receives their feedbacks. She shows them the possible drawings of the new interface and asks them to simulate trying it while thinking aloud. She summarizes the results and sets her decision. One of the teammates browses over the system reports and sees for each user experiment, which was conducted in the last two releases, what were the results and what were the implications on design. For each implication, he sees the development tasks that are related. We suggest that UCD management should be supported by an extension to a contemporary development environment in order to be used in a natural manner. This is elaborated in the next section. 3 The UCD Management Eclipse Plug-in We present the requirements of the first 5-week iteration of a project that is performed by six teams of six students each to develop the UCD management plug-in: 1. End-to-end UCD experiments: a. Experiments can be defined (date and time, assign users and teammates, store description, results, and conclusions) b. Experiments can be executed thus collecting data from User Experience (UX) according to the experiment c. Experiments can be viewed as per the results achieved d. Each group should select two kinds of experiments for implementations 2. Evaluation manager role-perspective: a. The evaluation manager role-perspective supports the management of the different experiments b. Experiments can be in different states c. The evaluation manager role-perspective supports i. Status view of the experiments ii. view of the current work items with respect to the experiments iii. UX results view 3. UI designer role-perspective: a. The designer role-perspective supports the UX-refactoring tasks (which are a special kind of work items) that emerge from the UX results b. A UX-refactoring task is associated with one or more UX results c. The UX results view reflects the status of the UX-refactoring tasks for example if a specific task is done or in progress it is marked in the UX results view 4. Work items can be created and assigned. 5. The system has one repository for its data. Two examples for possible interfaces as part of the Eclipse perspective were given to the development teams and are presented in Figures 1 and 2.

5 Figure 1. A screenshot taken from the interview interface Figure 2. A screenshot taken from the Evaluation Manager role-interface

6 The first example (Figure 1) is of an interview interface. During the planning session a task might be to perform interviews of the users of the system. The teammate, who has been assigned this task, selects the appropriate template from a set of templates provided by this content to use during the interview of users. The selected template will help her to interview the users in a standard or customized way. The second example (Figure 2) is of a UCD role interface. This content provides the definition of different roles, the related tasks, responsibilities, and the related artifacts. For example, it defines the role of the Evaluation Manager who can define and manipulate the UCD experiments. Following these set of requirements each of the teams suggests high level design to start with. Figures 3 and 4 present high level design of different parts taken from different teams. Figure 3 relates to the strategy of the experiment (a feature that was not specifically asked by the customer). Figure 3. High level design for attributes of an experiment The team describes the essence of the ExperimentStrategy interface as follows: Interface:yproj.core.model.experiment.ExperimentStrategy This interface should be implemented for each experiment in order to set its execution and data collection behavior. Covered Requirements: requirement 1 Extended Classes: java.lang.object Implemented Interfaces: Abstract Methods: Execute() executes the experiment Collect() collects the data from the executed experiment Figure 4 shows high level design of the Evaluation Manager perspective that includes as can be observed two views; one that controls the experiment parameters and one that provides with the results.

7 Figure 4. High level design for the evaluation manager perspective 4 Summary and Future Directions In this paper, we present an UCD management plug-in to better support UCD activities in software development processes. We base the capabilities suggested on use cases that were emerged when performing UCD activities with agile teams. In the future, we intend to continue developing the plug-in to support the use cases that are emerged, thus achieving a complete set of refined requirements for UCD management. Further, we intend to evaluate the plug-in with software development teams. Acknowledgements This research is supported by the DELOS Network of Excellence on Digital Libraries ( References Blomkvist, S. User-Centered Design and Agile Development of IT Systems. IT Licentiate theses, Department of Information Technology, Uppsala University (12, 2006). Blomkvist, S. Towards a Model for Bridging Agile Development and User-Centered Design. Published as a book chapter: Seffah, A., Gulliksen, J., and Desmarais, M., (eds.), Human-Centered Software Engineering Integrating Usability in The Development Process. Springer, Dordrecht, The Netherlands, Detweiler, M. Managing UCD within Agile Projects. ACM Interactions (May-June, 2007), Dix, A., Finlay, J.E., Abowd, G.D., and Beale, R. Human Computer Interaction, 3rd Edition, Prentice Hall (2003). Gulliksen, J., Goransson, B., Boivie, I., Blomkvist, S., Persson, J. and Cajander, A. Key principles for user-centered systems design. Behaviou & Information Technology, November December 2003, Vol. 22, No. 6, Hudson, W., Adopting User-Centered Design within an Agile Process: A Conversation. Cutter IT Journal, vol. 16, no. 10, (2003), Hwong, B., Laurance, D., Rudorfer A., and Song, X. User-Centered Design and Agile Software Development Processes, CHI workshop, 'Identifying 2004, Workshop Bridging Gaps Between HCI and Software Engineering and Design, and Boundary Objects to Bridge Them', Vienna, Austria, April 4, 2004.

8 McInerney, P., and Maurer, F. UCD in agile projects: Dream team or odd couple?. ACM Interactions, 12(6), 2005, Sharp, H., Rogers, Y., and Preece, J. Interaction Design: Beyond Human-Computer Interaction. 2nd Edition. Willey (2007).