Simulation in healthcare : design and assesment of chest auscultation software

Context and background: In health sciences, the evaluation system on real patients is confronted with ethical constraints, the patient should not be used as docimological material. Simulation as a pedagogical technique has been proposed as an alternative for the evaluation of acquisitions, especially in the framework of objective structured clinical examination. This technique faces difficulties due to the cost of simulators and the increasing number of learners. An evaluation software has been designed to limit the use of simulators; the objective of this study was to evaluate its reliability by comparing the scores obtained on the software vs the simulator (gold standard).

Methods: This is a reliability study of a chest auscultation assessment tool. It took place between September 2017 and February 2018, carried out after random sampling on IBM SPSS V.22 software on 45 students. The evaluation of the software was based on the ordinal concordance analysis (Cohen's Kappa test, and Kendall's correlation test) of the scores obtained by the students on the software compared to the simulator, considered as gold standard.

Results: For cardiac auscultation, the mean score was 90.6% of the total score on software vs 74.6% on simulator; Kendall correlation coefficients 0.15 (p non-significant) and Kappa correlation coefficients 0.15 (p= 0.116) revealed poor agreement according to Lander and Koch interpretation.  For pulmonary auscultation, the mean score was 95.6% of the total score on software vs. 88.4% on simulator; the correlation coefficients of Kendall 0.45 (p=0.014) and Kappa 0.64 (p=0.002) revealed a good agreement between the two assessment media.

Conclusion: Digital simulation appears to be an adequate solution for the assessment of acquisitions of chest auscultation for large numbers. The concordant results of the software concerning the lung section should encourage improvements in the cardiac auscultation component.