A Survey on Usability Evaluation in Digital Health and Potential Efficiency Issues

A Survey on Usability Evaluation in Digital Health and Potential Efficiency Issues is a research study published at HEALTHINF24 that aimed to collect and analyse data from 144 usability experts on their experiences with usability evaluation of digital health applications.

1. This file, "Usability Survey Paper Appendix A (Questionnaire)", contains the comprehensive set of questions and materials used in the usability survey discussed in the main paper. It serves as an essential resource for readers and researchers aiming to gain in-depth insights into the survey's questionnaire design, structure, and content. This appendix offers a detailed overview of the questionnaire, including the questions' wording, order, and categorizations, enabling an in-depth understanding of the survey's methodology and findings.
   • The survey had three different parts with a total of 19 questions including
     • 1) the introductory and screening part,
     • 2) the demographic part,
     • and 3) the main part asking usability-related questions (see survey questionnaires).
   • The questionnaire used in the study included both fixed-alternative questions (such as true/false, multiple choice, checkbox, and rating scale) and open-ended free-text responses. It incorporates an initial set of response options for the fixed-alternative questions, which were primarily derived from the relevant literature and grey literature. Almost every question included open-text fields to accommodate a broader range of responses, allowing participants to share answers not listed in the fixed alternative questions.
   • The introductory and screening part includes survey information and screening them based on a non-leading mandatory screening question to determine whether volunteers meet the eligibility criteria. 
   • The demographic part consists of eight questions designed to know participants better regarding their demographics and previous experiences in assessing and ensuring the usability of digital health applications (see Part II of survey questionnaire). The questions asked participants about their gender, age, job position or title, and their experiences regarding evaluating the usability of applications in the digital health sector. They were also asked about the types of digital health systems/services/technologies they had evaluated for usability.
   • The main part contains ten usability evaluation related questions, with one attention check question placed in the middle of the survey. This section focuses on usability evaluation tools, methods, and approaches to understand how usability experts assess and ensure the usability of digital healthcare software. Participants were also asked about the usability characteristics that are covered during the usability evaluation in digital healthcare. Furthermore, another sub-part of this section explores the benefits of using tools during usability evaluation, as well as the overall perceived benefits and challenges encountered during conducting usability evaluation of digital health apps.
2. Appendix B, titled "Usability Survey Paper Appendix B (Detailed Results in Tabular Form)," provides an exhaustive compilation of the results obtained from the usability survey detailed in the main paper. The file contains organized, tabulated data offering insights into participants' responses, practices, and experiences. Each table is carefully structured to ensure clarity, making the data accessible and interpretable for subsequent analysis, review, and comparison.
3. The quantitative dataset (filename: Usability Survey Paper Dataset.xlsx) includes the responses of usability experts to questionnaires about their demographics, experience with usability testing, and the frequency of use of different usability evaluation methods. This dataset also includes quantitative data on the ranking of the importance of different aspects of usability evaluation, such as ease of use, efficiency, and satisfaction. The qualitative dataset also includes the responses of usability experts to open-ended field of survey questions.