D4.5 ARETE Use Scenarios: Analysis of Educational Technologies Usage in Situ – UPDATE

This deliverable D4.5 reports the evaluation methods and results that WP4 has implemented and obtained since the submission of D4.2 in M18. In close collaborations with WP3, WP5 and WP6, the HCI team of WP4 endeavoured to provide timely feedback to the respective teams for improving the interaction quality of the design artefacts generated for the four Pilots. Utilising the established human-centred design methods, we have identified usability problems (UPs). For each UP, we assigned an importance level (low, medium, high) for it to be fixed and proposed recommended modifications for the development team to consider.

The table below presents an overview of the evaluation studies undertaken. The design artefacts for each Pilot were at different development stages when under evaluation. The number of usability problems (UPs) varied with the complexity of the software package involved. Specifically, the Pilot 2 package comprised five components - two interactive apps, two workbooks and one map – accounting for a higher number of UPs, albeit the majority was of low importance. Encouragingly, the revised versions of the design artefacts evaluated were found to be more usable and pleasurable to use.

Furthermore, WP4 explored the innovative approaches to evaluating eXtended Reality (XR) applications with the wider research community. A workshop was held where emergent XR devices (e.g., smart glasses) and innovative methods (e.g., multisensory fusion) were explored.