Usability: An Introduction Joseph Kannry, MD Lead Technical Informaticist Mount Sinai Health System Professor Medicine Icahn School of Medicine at Mount Sinai
Defining Usability Usability has been studied in computer science and information technology for decades HCI Usability is best defined as the ease of use of a system by humans Usability = Usable + Useful Usability = Usable + Useful + Safe
Usable Information Systems How easy it is to use? How efficient is it? How easy is it to learn? How enjoyable it is to use?
Useful Information Systems Does it help you in your day? Physicians look for three things when using HIT: Helps with my patients Improves patient care (my patients) Makes job easier 1.Weiner M, Gress T, Thiemann DR, Jenckes M, Reel SL, Mandell SF, et al. Contrasting views of physicians and nurses about an inpatient computer- based provider order-entry system. J Am Med Inform Assoc. 1999;6(3):234-44.
Usability, Satisfaction, and Efficiency: Kannry and Murff Using a standardized instrument, QUIS, assessed user satisfaction with two CPOE systems House staff were dissatisfied with the commercial system mean score of 3.67 (95 percent confidence interval [95%CI], 3.37-3.97). CPRS had a mean score of 7.21 (95% CI, 7.00-7.43) House staff satisfied with the system
Usability, Satisfaction, and Efficiency: Kannry and Murff Satisfaction was most strongly correlated with the ability to perform tasks in a "straightforward" manner. Murff HJ, Kannry J. Physician satisfaction with two order entry systems. J Am Med Inform Assoc. 2001 Sep-Oct;8(5):499-509. Similarly noted Sittig DF, Kuperman GJ, Fiskio J. Evaluating physician satisfaction regarding user interactions with an electronic medical record system. Proc AMIA Symp. 1999:400-4.
Usability: A nice concept but Why Does It Matter?
Usability Matters Patient Safety User Satisfaction
Usability and Patient Safety
Usability: Recommended Or Required By Task force on EHR 2020 recommends focusing on Usability Payne TH, Corley S, Cullen TA, Gandhi TK, Harrington L, Kuperman GJ, et al. Report of the AMIA EHR 2020 Task Force on the Status and Future Direction of EHRs. J Am Med Inform Assoc. 2015 May 28. Usability issues can result in unintended consequences and patient harm Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. Journal of the American Medical Informatics Association. 2013;20(e1):e2-e8.
Usability Causing Patient Harm Alphabetizing dosing choices for medication dosing Eight, four, five, one, nine, six, seven, three, two No known allergy and no known drug allergy + multiple allergies No allergy button and there are known allergies Kannry J, Kushniruk A, Koppel R. Meaningful Usability: Health Care Information Technology for the Rest of Us. In: Ong K, editor. Medical Informatics: An Executive Primer. 2nd ed. Chicago: HIMSS; 2011.
Usability Recommended Or Required By NIST (National Institute of Standards and Technology) Recommends EHRs should be tested for Usability Uses Nielsen s 10 Heuristic Rules for Usability Nielsen J. 10 Usability Heuristics for User Interface Design 1995 Available from: http://www.nngroup.com/articles/ten-usabilityheuristics/
Nielsen s 10 Heuristic Rules Visibility of system status Match between system and the real world User control and freedom Consistency and standards Error prevention
Nielsen s 10 Heuristic Rules Recognition rather than recall Flexibility and efficiency of use Aesthetic and minimalist design Help users recognize, diagnose, and recover from errors Help and documentation 10 Heuristics for User Interface Design: Article by Jakob Nielsen http://www.nngroup.com/articles/ten-usabilityheuristics/
Usability Recommended or Required By ONC(Office of the National Coordinator) for both Stage 2 and 3 Certification Stage 2 requires a User Centered Design (UCD) process and Summative Usability on Sections of Application Stage 3 requires testing for tasks Effectiveness Efficiency Satisfaction
Usability Recommended or Required By ONC Tasks 1. Computerized Provider Order Entry System (CPOE) ( 170.314(a)(1)); 2. Drug- drug, drug-allergy interaction checks ( 170.314(a)(2)); 3. Medication list ( 170.314(a)(6)); 4. Medication allergy list ( 170.314(a)(7)); 5. Clinical decision support ( 170.314(a)(8)); 6. Electronic medication administration record ( 170.314(a)(16)); 7. Electronic prescribing ( 170.314(b)(3)); 8. Clinical information reconciliation ( 170.314(b)(4)). http://www.healthcareusability.com/article/170314g3-safetyenhanced-design-and-meaningful-use-stage-two-and-three
Usability: How Well Do the Vendors Do Today on UCD (User Centered Design)? 11 Vendors 3 Categories: Well-Developed, Basic, and Misconceptions Staff: 0 to >30
Usability: How Well Do the Vendors Do Today on UCD (User Centered Design)? Challenges Contextual workflow Participants Vendor leadership Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the usercentered design processes of eleven electronic health record vendors. Journal of the American Medical Informatics Association : JAMIA. 2015 Jun 6.
Usability and Use icpr was a (RCT) (Randomized Control Trial) looking at the integration Clinical Prediction Rules in an EHR icpr used multiple rounds of usability testing and iterative development Users adopted the intervention (57.4% of users) and accepted the recommendation (42.7% of users) *Kannry, J, McCullagh L, Kusniruk, A, Mann, D, Edonyabo, DA Mcginn T. Framework for Usable and Effective Clinical Decision Support: Experience from the icpr Randomized Clinical Trial. egems. Vol. 3 (2015). Iss. 3
Concluding Thoughts Usability = usable + useful Profound effect on user satisfaction and patient safety Recommended or required by 2 AMIA Task Forces, NIST, and ONC Every Enterprise should have a Usability Testing Laboratory
Concluding Thoughts Vendor Testing variable Multiple papers point to a lack of standard scripts and workflows to test usability on All agree usability is critical for safe and continued used of EHRs.