Abstract: PO1567
Human Factors Impact on the Development of Software as a Medical Device (SaMD): A Case Study Using the System Usability Scale (SUS)
Session Information
- Cystic Kidney Diseases: Emerging Concepts, Biomarkers, and Clinical Trials
October 22, 2020 | Location: On-Demand
Abstract Time: 10:00 AM - 12:00 PM
Category: Genetic Diseases of the Kidneys
- 1001 Genetic Diseases of the Kidneys: Cystic
Authors
- Hossain, Mosaddeq, Otsuka Pharmaceutical Development and Commercialization, Inc., Princeton, New Jersey, United States
- Dixon, Mary, Innovenn, Inc., Madison, Wisconsin, United States
- Lutz, Annie, Innovenn, Inc., Madison, Wisconsin, United States
- Horvath, Jeff, Balanced Experience, LLC, Madison, Wisconsin, United States
Background
Autosomal Dominant Polycystic Kidney Disease (ADPKD) is an inherited, progressive, cystic kidney disease and is the fourth leading cause of end-stage renal disease (ESRD). Total kidney volume (TKV) is the most relevant imaging biomarker for tracking and predicting the natural course of ADPKD. Accurate prognostic tools may help predict outcomes and optimize clinical management to slow the loss of renal function. The ADPKD Progression Management (APM) System is a web-based, clinical-decision support software that offers a consistent method for estimating TKV and aids in the prediction of likely risk of progression. The system helps health care providers (HCPs) automatically calculate TKV, project likely TKV growth, and track eGFR and TKV changes over time. The APM System was evaluated for perceived ease-of-use and user satisfaction utilizing the industry standard System Usability Scale (SUS). The SUS is a simple, reliable tool consisting of 10-items with five response options, anchored by “Strongly Disagree” and “Strongly Agree”; the calculated score ranges from 0-100.
Methods
The SUS was used in 4 human factors studies of the APM System. Participants included nephrologists and radiologists who completed test cases using mock data. Participants had no prior experience with the software and received no training on the system. Participants entered clinical information, utilized the automated image measurement to calculate TKV, and generated the automated statistical calculation of predicted growth of TKV, a marker of disease progression. Participants then completed a SUS questionnaire. In each study, a global SUS score was calculated; a total of 79 participants contributed to the global SUS scores: 37 nephrologists, 28 radiologist and 14 nephrologist/radiologist support staff.
Results
APM System received the following SUS scores in studies one through four: 72, 77, 70*, and 80. The SUS score showed an 8 point increase from the first study to the fourth study.
*New functions (e.g. Consultation workflow) introduced.
Conclusion
The SUS results demonstrate the impact of iterative improvements in the design and usability of the APM System. The SUS global scores provide evidence that the perceived usability is above average and comparable to the average SUS score for the top 10 apps across iPhone, Android and tablets.