Modelling Usability Capability Introducing the Dimensions

Transcription

1 Lecture Notes in Computer Science 1 Modelling Usability Capability Introducing the Dimensions Timo Jokela 1 and Pekka Abrahamsson University of Oulu Address: P.O. Box University of Oulu, Finland {timo.jokela, pekka.abrahamsson}@oulu.fi 1 Also: Nokia Mobile Phones, Oulu, Finland Abstract. Usability capability is a characteristic of a development organization that predicts the level of usability the development projects are capable of achieving. Our experiments with the existing usability capability models indicate that current process assessment methods do not discover all relevant problems that might impede effective user-centered design (UCD) in development organizations. We propose an enhanced model where the usability capability is analyzed from three dimensions: user-centered infrastructure, implementation of user-centered practices in development projects, and business management commitment to usability as a competitive asset. 1 Introduction This paper concentrates on usability capability models. In this section, we will describe the related basic concepts (i.e. usability, user-centered design, and usability capability) and give motivation to this research effort. Usability is defined as one of the main software quality attributes in the standard ISO 9126 [12] as follows: 'The capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions". Another frequently referred definition of usability is that one in the standard ISO [12] where usability is defined as follows: 'The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use". How to design usability? The prevailing paradigm is the user-centered design (UCD) which is "an approach to interactive system development that focuses specifically on making systems usable" (ISO 13407). Effective user-centered design leads to usable products, as illustrated in Fig. 1.

2 Lecture Notes in Computer Science 2 Effective user centered design leads to Usable products Fig. 1. There is a positive relationship between usability and the performance of UCD In practice, the position of UCD in the development organizations is problematic. When we examine software intensive systems and products that are in the market we can observe that many of those represent poor level of usability. This seems to be the case also even with companies who have dedicated resources to UCD, and where usability actions have been going on for years. Some of the products or systems that these companies bring to the market may represent good usability, while some other products (of the same company) may fail in achieving usability either partially or even seriously. A number of articles (e.g. [1, 2, 4, 24]) report the problematic position of user-centered design. It is also a topic of many presentations and panels in conferences and seminars: there was a tutorial [3], a panel [21] and an interview of Don Norman and Janice Rohn [19] at CHI'99. Moreover, authors share these experiences since one of the authors has been a practitioner in a large development organization for several years. We define usability capability as follows (Fig. 2): The characteristic of a development organization that determines its ability to consistently develop products with high and competitive level of usability. 1 Usability capability determines Effectiveness of user centered design Usable products Fig. 2. Usability capability of a development organization determines its ability to effective perform user-centered design. In development organizations with high level of usability capability, user-centered design is effective and efficient which leads to usable products or systems. Low usability capability level means either non-existent or ineffective user-centered design which results to poor usability. How to improve the usability capability of development organizations? In the world of software development process assessment models [14, 18, 20] have been introduced to guide improvement efforts of software processes. By performing process assessments, one can identify the strengths and weaknesses of an organization in software development, and use this information to focus improvement actions. 1 Author refers here to the definition of capability in Trillium [23]: 'The ability of a development organization to consistently deliver a product or an enhancement to an existing product that meets customer expectations, with minimal defects, for the lowest life-cycle cost, and in the shortest time.'

3 Lecture Notes in Computer Science 3 Analogically, in the world of user-centered design, usability capability models have been introduced. Through usability capability assessments, one can get guidance for improvement actions to raise the level of usability capability of development organizations, as illustrated in Fig 3. Usability capability assessments guide Improvement lead to actions Improved usability capability Fig. 3. Usability capability assessments guide improvement actions of usability capability. The objective of this paper is to propose dimensions for an enhanced usability capability model. In the following section, we will give a brief overview of the existing usability capability models. Then we will report our experiences on assessing usability capability with one existing model: INUSE process assessment. Based on the assessment experience, our experience as practitioners, and reports from other practitioners, we then analyze the 'essence' of usability capability: what exactly are those elements of organization that have impact on usability capability and should be covered in a respective assessment. In the final section, we present a preliminary new model. The paper is concluded with a summary of the results and suggestions for further work in the area. 2 Overview of the Existing Usability Capability Models In this section we describe briefly six existing usability capability models. Even though they are called with different terms, they all address basically the same theme: a description of the ability of organizations to design usable products or systems. Three of the usability capability models Trillium [23], Philips [10], and INUSE Usability Maturity Model: Processes (INUSE PROC) [1] have their origins in the process assessment models of software development. Trillium and Philips refer to CMM [20] while INUSE PROC refers to ISO [14]. The rest of the models Ehrlich & Rohn [7], IBM [8], and INUSE Usability Maturity Model: Human-Centredness Scale (INUSE HCS) [6] are based on different approaches (than process assessment) we call them generic capability models. Fig. 4. The work by Ehrich and Rohn cannot be regarded as a 'model' it presents the stages only with a brief description (10 to 20 lines of text per stage). It is included here since it is one of the early efforts to define usability capability and it is presented in a well-known book [7]. They define four different stages for usability capability: skepticism, curiosity, acceptance, and partnership. Trillium is a process assessment model for development of telecommunication products. It is a large model covering a wide variety of different processes and practices of product development, including also a number of usability practices. It is well

4 Lecture Notes in Computer Science 4 documented and published as a public domain, being therefore downloadable from the web. PROCESS ASSESSMENT MODELS GENERIC CAPABILITY MODELS Trillium Philps INUSE UMM Process Ehrlich & Rohn IBM INUSE UMM Human Centredness Fig. 4. Usability capability models can be categories into two groups Another process assessment model is developed by Philips. They refer to their assessment model as 'Humanware process assessment'. There is a one position paper available about the approach. It contains short definitions for ten 'humanware' processes. A significant feature of the Philips model is that it covers a large set of management related processes (labeled as key process areas). The INUSE PROC is a well-documented process assessment model that is based on the format of the software process assessment model defined in ISO It defines seven processes, five of which are derived directly from the standard ISO The IBM model is rather large: it covers organization, skills and process aspects. There is only a limited documentation available about the IBM model, without any references to other models. The only documentation publicly available is the one delivered at the CHI'95 in a special interest group session. The INUSE HCS is based e.g. on the IBM model and that of ISO It is, therefore, regarded to be a good reference. It has only one dimension human-centredness. The model is intended to be used "in the assessment of the human-centredness of an organization or department". It defines six increasing levels of maturity of humancentered processes from 'unrecognized' to 'institutionalized'. 3 Experiences with Process Assessments We carried out experiments with INUSE process assessment model in two development organizations. One of the organizations has more than ten years of experience in user-centered design being a pioneer in Finland while the other had only 2 to 3 years of experience in user-centered at the time of assessments were carried out. In this paper we provide a summary of the results. A more detailed report on the experiments can be found in [16].

5 Lecture Notes in Computer Science Description of INUSE Usability Process Assessment Model The INUSE PROC is a well-documented process assessment model. It is based on the format of the software process assessment model defined in ISO It defines seven processes, five of which are derived directly from the standard ISO The aim of the assessment was to find out the level of maturity of the assessed part of the organization, and to test the adequacy of the used model. In both cases the focus of evaluation was at the level of development projects. The corresponding units or organizations were not evaluated as a whole. A new process category called Human-Centered Design, consisting of seven human centered processes (HCD.1 to HCD.7), was added to the ISO model. The structure of the model is illustrated in Figure 5. The base practices belonging to one of the processes (HCD.4: Understand and Specify the Context of Use) are shown as an exa mple. ISO Categories Customer Supplier Process Category Engineering Process Category Support Process Category Management Process Category Organization Process Category Human-Centered Process Category (INUSE addition) HCD.1 Ensure HCD content in system strategy HCD.2 Plan the human-centered design process HCD.3 Specify the user and organizational requirements HCD.4 Understand and specify the context of use - clarify system goals - identify user s tasks - identify user s attributes - identify organizational environment - identify technical environment - identify physical environment HCD.5 Produce design solutions HCD. 6 Evaluate design against requirements HCD.7 Facilitate the human-system implementation Fig. 5. The structure of the INUSE process model The central reference for this structure has been the use of the process structure defined in the standard ISO as the core organizing principle (the processes HCD.2-6 are taken directly from ISO 13407). 3.2 Experiences with Process Assessments The model was found to be working and producing meaningful and useful results especially if the organization has a long tradition in user-centered design. In one case,

6 Lecture Notes in Computer Science 6 the assessment generated a meaningful and productive discussion about the results and potential improvements of the processes in the feedback situation, and this discussion relatively rapidly converged into suggestions for practical improvements. However, we found that it is evident that assessing the capability of processes is not always the best choice. We have identified a number of different problems that the process assessment does not address directly. Usability practitioners have acknowledged these problems as being serious ones. Such problems are, for exa mple: the late participation of the usability specialists in development projects (major design decisions that have impact on usability are made before their involvement) some critical user interface decisions are done outside of project control not all key players respect the usability skills and may misunderstand their role (words of one key player: "I represent end users in this organization") some key players use usability was as a buzzword rather than in its true meaning ("we are good in usability, because our user interfaces have style and are good looking") user-centered design activities are carried out but they do not always have impact on decisions the position of usability specialists in the project is invisible, they are not among decision makers some project managers do not have any commitment to usability (words by a project manager: "I don't want any more usability work in this project!") due to the positive image of 'usability', the term is used as a justification even to designs that actually have significant usability problems implementation of user-centered design in projects is supply-driven, initiated by UCD resources (not by project management) if some user task is found to be difficult to use with a user interface design, it is claimed to be the customer's fault ("customers are wrong when they want users to perform these tasks") Others report similar problems also. For example, Don Norman one of the most renowned gurus in the HCI field - said in an interview at CHI'99: "User testing is low level work, other people make decisions. I don't want to be called usability expert.", "Important question: peers or resources? Usability people are normally resources", and "Company thinks that when usability tested then it is done." In software development, the critical problems are the projects being late and over budget. As written in the book about CMM [20]: "Organizations are now realizing that their fundamental problem is the inability to manage the software process. In many organizations, projects are often excessively late and over budget, and the benefits of better methods and tools cannot be realized in the maelstrom of an undisciplined, chaotic project." In user-centered design, on the contrary, Earthy describes an immature stage as follows [6]: "The need for a human-centered process is not recognized. If systems are received with varying degrees of satisfaction by their end users this does not cause concern". Ehrlich and Rohn [7] state that "UCD is viewed with some skepticism be-

7 Lecture Notes in Computer Science 7 cause it is unclear what benefits it will bring. There is fear that UCD will lengthen the product development cycle. If a UCD expert is involved at all, he or she is brought in late in the development cycle." The critical problem related to UCD is not the management of the process itself but to have UCD processes existing and making them effective. In summary, we believe that usability capability assessments should cover also other aspects than user-centered design processes. The limited focus of process assessment is acknowledged also in [20]: "The CMM is not comprehensive; it only touches on other, non-process factors, such as people and technology that affect the success of software projects. The mature organization addresses all issues essential to a successful project, including people and technology, as well as process." 4 Elements of Usability Capability: What Is Required for Designing Usability? The findings reported above inspired us to start thinking of the fundamentals of usability capability. What are those organizational elements that have impact on usability capability? When usability capability is analyzed, what should be examined in addition to the user-centered processes? In this section, we present conclusions about the aspects that the usability capability assessment should cover in a development organization. 4.1 Performance of User-centered Design at Project Level In a typical product development organizations, products are developed in development projects. Usability is a quality attribute of a product that is dependent on the extent and performance of UCD activities in a specific development project. High performance UCD is composed of a number of different organizational elements. The development process should incorporate the activities and follow the principles of user-centered design. ISO defines a set of principles and activities of UCD through which usable systems or product can be created. The principles are multi-disciplinary teamwork, active involvement of users, iteration of design solutions, and appropriate allocation of functions between the system and users The activities defined by ISO are: understand and specify the context of use, specify the user and organizational requirements, produce design solutions, and evaluate design solutions against requirements, as illustrated in Fig. 6. The figure illustrates also the principle of iteration. End-user involvement should take place, as applicable, throughout the activities.

8 Lecture Notes in Computer Science 8 Understand & specify the context of use Evaluate designs against requirements System meets specified user & organisational requirements Specify the user & organizational requirements Produce design solutions Fig. 6. Activities of UCD UCD activities should be performed in a high quality way in order to produce valid data for the product design. For that, there needs to be a set of different resources in the organization. First, there should be UCD skills available in the organization (i.e. no meaningful UCD activities can performed without skilled resources). A key resource is usability specialist with good knowledge of user-centered processes, methods and tools. In addition, the UCD activities should be carried out with appropriate procedures, methods, tools and technology. An important aspect is that the results from user-centered activities are used so that they have an impact in the design of the product. Experience has shown that even though UCD activities have succeeded in providing valid data, these results have been ignored in the decision making concerning the project. Another elementary factor to the performance of UCD is that the development team is committed to UCD. Without awareness of benefits of usability, software designers may feel frustrated because UCD activities seem to take long time in the early phases of a project. Nothing concrete (i.e. in their eyes) seems to happen. Effective usercentered techniques like paper prototyping may seem inefficient because it may postpone traditional software engineering activities. In summary, effective UCD in development projects requires a number of organizational elements, as illustrated in Fig. 7: the incorporation of UCD activities and principles in the project life cycle, people with new skills, effective and efficient usercentered methods and tools, project team awareness about usability and commitment to UCD, respect of UCD work by project staff, and finally the utilization of results from user-centered activities in the design process. At a poor capability level there are no user-centered activities carried out in a project. Or if there are some, they are carried out in an inappropriate and inefficient way. In an ideal scenario UCD has a significant role, efficient methods are used, project staff

9 Lecture Notes in Computer Science 9 respects UCD work, and results are used in the design. A capability model should be able to make a distinction in performance between different projects. DEVELOPMENT PROJECT Usability Culture Impementation of Principles and Activities of User Centered Design UCD Resources Usable products Fig. 7. Successful UCD requires many elements to be in place in development projects. 4.2 Mechanism for Ensuring Routine Use of UCD As stated earlier, the usability of a product is dependent on the performance of UCD carried out in development project and the usability capability of a development organization determines the extent to which UCD is implemented throughout the organization in the different product development projects. Keeping the schedule is typically the most important success factor set to development projects. Acknowledging this, the incorporation of UCD sets new requirements for development projects: new skills are required, new activities to be performed, and new training to be organized. Thus, from the viewpoint of development manager, the activities of UCD may mean new workload to the project, which again means new risks in meeting time constraints set for the project. One needs to carry out activities that were not done earlier, and one needs to recruit people with special new skills. Hakiel [6] has argued: "UCD as such has a little value to a development manager It adds to the budget and involves new activities whose role in the development process in not always evident". Our experiences are similar. Long term benefits, however, show that UCD leads to decrease in development costs and shortens the development cycle. This occurs because UCD strengthens the quality of requirements definition phase thus decreasing the number of late changes in

10 Lecture Notes in Computer Science 10 requirements. However, it may not be easy to convince these benefits to a development manager. There is a need to make organizational changes as well: a team of people with a new role should be allocated to the projects. If UCD is a novel issue in the organization, awareness and commitment has to be established. This can be solved through training, which again might add extra burden to the project since it takes time and resources. While time and resources are typically the bottleneck, finding place for training in projects may turn out to be a problem. In an ideal case, UCD is accepted as part of development process by the decision of the development manager (i.e. if (s)he understands the value of usability). The inherent problem here is as follows: how to have UCD to play an effective role in a product development project when the issue of usability isn't in the set of important objectives (like keeping the schedule) of the development manager? How to motivate an organization for this kind of change? Hakiel [11] addresses this issue. His conclusion is that in order to make UCD routine, there should be demand of usability of products from the business management. Business management should view usability as an important competitive asset, and perform tangible actions that create demand for UCD in the development projects. We find this can be achieved by, for example, defining a clear and measurable usability objectives for the projects, and having a reward structure in place supporting the project team to meet these objectives (Fig. 8). We have had promising experience about the use of incentives in development projects: usability objectives were clearly met and even exceeded (see details in [15]). At a poor usability capability level the organization may be able to perform UCD successfully in some individual projects while many others may lack it completely. A capability model should be able to identify the extent to which the organization systematically uses UCD.

11 Lecture Notes in Computer Science 11 Management Demand on Usability Incentives Usability objectives User Centered Design in Development Projects Usable products Fig. 8. Management Demand on Usability Leads to Effective User-centered Design 4.3 Keeping Infrastructure for UCD at Competitive Level Considering the long term positive effects, it is not enough for high usability capability level organization that it performs UCD consistently, and that it incorporates UCD into development projects an organization wide practice. Organization has to be able to maintain its ability to perform competitive UCD also in the future. In a true high usability capability level organization the senior management is concerned with the ability of the organization to meet the competition in usability of its products, and performs appropriate actions to maintain the ability for performing UCD at a competitive level. This can be achieved by following the status of organization's products' usability compared with the ones of competitors, and by concentrating on activities that ensure that UCD resources maintain their competitive level. This includes issues as follows: the state-of-the-art methods are used, and appropriate skills remain in the organization, there are high level resources for effective and efficient implementation of UCD in projects, and finally, UCD is acknowledged as one of the core competencies of the organization. A capability model should be able to make the distinction between different organizations by identifying the extent to which the organization pursues maintaining their ability and resources in performing UCD at a competitive level. 4.4 Summary: Organizational Elements of User-centered Design In summary, the main requirement for usable products is that UCD takes effectively place in development projects. For that, there should be two basic structures in place: infrastructure for UCD (skills, methods, tools, awareness and commitment to UCD from personnel), and business management commitment towards usability (management sets tangible requirements to projects, follows the situation in the market, and pays

12 Lecture Notes in Computer Science 12 attention maintaining the infrastructure at competitive level). This is illustrated in Fig. 9. Competitors Market follow-up Usablity requirements Business Management commitment Improvement requirements Products DEVELOPMENT PROJECTS User centered design User centered design infrastructure Resources Fig. 9. Effective and efficient user-centered design requires infrastructure and commitment from business management 5 A Proposal for an Enhanced Usability Capability Model In this section, we describe the dimensions for an enhanced usability capability mo d- els. The model is based on the discussion above and analysis of the existing models. A more detailed discussion of the models can be found in [16]. 5.1 Dimensions of UCD Capability We suggest that the following aspects should be analyzed when assessing the usability capability: business management commitment, UCD infrastructure, and effectiveness of UCD in development projects (Fig. 10).

13 Lecture Notes in Computer Science 13 Business management commitment to usability as comptetitive asset Infrastructure: resources for user centered design Implementation of user centered design in development projects Fig. 10. Dimensions of Usability Capability Infrastructure. The extent to which the organization has resources to effectively and efficiently plan and implement high quality UCD in development projects. The elementary element to make UCD effective is to have UCD skills in the organization. We need enough skills (i.e. qualified UCD practitioners) so that the UCD support can be supplied to development projects. Another elementary element is the technology to implement UCD activities (e.g. prototyping tools and usability labs). Skills and technology are the basic infrastructure. While skills and technology are sufficient for making UCD effective in a development project, they are not enough to make UCD efficient. To make the UCD work efficient (i.e. to make it happen quickly) we need guidelines for how to plan and implement UCD work. Moreover, we need awareness and commitment to usability from the project team. If usability and UCD are new issues, it will take time and resources to give the required usability training. Summary: UCD Infrastructure determines the level how effectively and efficiently UCD can potentially be performed. However, the existence such infrastructure alone does not guarantee effective UCD in development projects. For that, we need to consider the next dimension. Implementation of UCD. The extent to which UCD is planned and implemented in development projects in terms of effectiveness, efficiency and quality. Usability is a quality characteristic of a product that is determined by the extent and quality of UCD that takes place in a development project. If the principles and activi-

14 Lecture Notes in Computer Science 14 ties of UCD are implemented in the project in a high quality way, it can be assumed that the outcome of the project is a product with high level usability. The necessary requirements for high quality implementation of UCD are the existance of the basic UCD infrastructure (i.e. skills and technology), and their availability to the project. All that is needed to plan and implement high level UCD are project management commitment, time and people resources. Other enablers (i.e. guides and project team awareness of usability) make the planning and the implementation of the UCD more efficient. Summary: UCD Effectiveness determines in the end the usability of the product that a project develops. The usability is determined by the extent and quality of the UCD work that is carried out in the project. The quality cannot be any higher than the UCD Infrastructure can provide. Business Management Commitment to Usability. The extent to which business management has demand for usable products and commitment for development of UCD infrastructure. If this dimension is not existing, the role of UCD in development projects is increasingly project dependent (i.e. it depends solely on the attitude and commitment of the project management and the team). If usability is not in the objectives of project managers or designers, 'selling' UCD to a project may prove to be a difficult task. It is easily seen just as an add-on to the project, which may threaten the project schedule. It is important to understand that true commitment of management to usability is more than talking about usability with a positive tone. Commitment has to visible through manager's behavior: management sets clear usability objectives to projects, follows the competitive situation in the market place from the usability point of view, and performs actions to ensure the competitive performance level of UCD infrastructure. The basis for the commitment of business management is the awareness of the business benefits of usable products. Summary: Business management commitment to usability determines the usability capability of the organization in the long run. If the commitment is lacking, there is a great variation in the UCD Implementation even if certain UCD Enablers are existing. In addition, the lack of business management commitment also keeps the role of UCD personnel unvisible. 6 Conclusions and Further Work In this paper we have introduced the dimensions for an enhanced usability capability model. The model is based on our experiences on usability process assessment, our professional experience as user-centered design practitioners, experience reports from different sources, and analysis of existing capability mo dels. The model proposes that usability capability should be analyzed from three main dimensions: the level of business management commitment, the level of the user-

15 Lecture Notes in Computer Science 15 centered design infrastructure, and the level of the capability of user-centered design processes. A distinctive feature in our proposal is that it covers wider aspect of the organization than currently used process assessment models. The model proposed covers issues including skills, empowerment, awareness, attitudes, technology, culture that are not usually explored by standard software process assessments. The model is at its preliminary state. The dimensions should be further explored in detail. We plan to further develop and experiment the model in the near future. A national research project KESSU, funded by the Finnish national funding organization TEKES, will give a good platform for pursuing this work further. References 1. Aucella, A Ensuring Success with Usabilily Engineering. In Interactions, May+June, pp Bloomer S. & Croft R Pitching Usabilty to Your Organization. Interactions. Nov Dec 1997, pp Bloomer S., Wolfe S. Successful strategies for Selling Usability into Organizations. CHI 99 Extendend Abstracts, pp Browne In Politics of Usability 5. Earthy, J. (1998a), Usability Maturity Model: Processes (INUSE Deliverable D5.1.4p). Available at (December 1999) 6. Earthy, J. (1998b), Usability Maturity Model: Human Centredness Scale. INUSE Deliverable D5.1.4s. Available at (December 1999) 7. Erlich K, Rohn J. (1994), Cost Justification of Usability Engineering: A Vendor's Perspective. In Bias R., Mayhew D. Cost-Justifying Usability. Academic Press, pp Fellenz, C Introducing Usability into Smaller Organizations. In Interactions, Sept+Oct, pp Flanaghan G.A. (1995), IBM Usability Leadership Maturity model (self-assessment version). Distributed at CHI Workshop. 10. Gupta, A. (1997), The Humanware Process Improvement Framework: Interfacing Usercentered Design and the Product Creation Process at Philips. Position paper delivered at HCPIG meeting in Teddington, UK. 11. Hakiel, S. (1999), Sufficient and Necessary Conditions For Routine Deployment Of User- Cnetred Design. Sufficient and necessary conditions for routine deployment. IEE colloquium on Making User-Centered Design Work in Software Development 12. ISO Software product quality. International Standard. 13. ISO (1999), Human-centered deisgn processes for interactive systems. International Standard. 14. ISO (1998), Software Process Assessment. International Standard. 15. Jokela, T., Pirkola J. (1999a). Using Quantitative Usabilily Goals in the Design of a User Interface for Cellular Phones. Proceedings of INTERCT '99. Volume II. pp

16 Lecture Notes in Computer Science Jokela, T. (2000). Usability Capability Models Review and Analysis. To be presented at HCI 2000 conference. Sunderland. UK. 17. Kuutti, K., Jokela T., Nieminen, M., Jokela, P. (1998), Assessing human-centered design processes in product development by using the INUSE maturity model. Proceedings of Analysis, Design and Evaluation of Man-Machine Systems. Kyoto, Japan. 18. Kuvaja, P. et al. (1994), Software Process Assessment and Improvement. The Bootstrap Approach. Oxford: Blackwell. 19. Norman, D. (1999), Organizational Limits to HCI. Interview session at CHI'99. Pittsburgh, USA 20. Paulk, M. C. & al (1995), The Capability Maturity Model: Guidelines for Improving the Software Process. Addison-Wesley. 21. Rosenbaum S,. What Makes Strategic Usability Fail? Lessons Learned from the Field. CHI 99 Extendend Abstracts, pp Schneider C. (1996). Using Paper Prototyping to Manage Risk. Software Design and Publisher Magazine 23. Trillium. (1994), Model for Telecom Product Development & Support Process Capability. Release 3. Bell Canada Available at (December 1999) 24. Tudor, L Human Factors: Does Your Management Hear You? In Interactions, Jan Feb 1998, pp