Development of a Mixed Reality Game for Simulation Based Education

Mixed reality allows for new ways for learners to enrich their learning experience. By enabling the use of 3D virtual content in a tangible space, it offers a more immersive digital learning environment than traditional on-screen methods. This paper reports on the use of the Mixed Reality (MR) Microsoft HoloLens to create a mixed reality Vocational Educational Training (VET) serious game.
The main challenges of this investigation are the use of tangible objects as an interface to a virtual learning environment, which is achieved through low-latency, real-time tracking, allowing for believable interaction with virtual content. This is examined through the creation of a stone cutting application incorporating the use of a 9’ angle grinder for training college-level stonemasons, whereby the learner attempts to cut a check into a virtual stone block using a real grinder. Cyber-physical systems (CPS) are utilised to realise this as part of the digital pedagogy. Currently, the vocational training is constrained by organisational health and safety regulations which do not allow students to use power tools such as an angle grinder within the college. By using a virtual stone block, a ‘real’ physical grinder as a tangible interface can be used without the associated safety risks.
The application has been co-designed and developed in association with several further education (FE) colleges in Scotland and has received appraisals from teachers and learners. On-site demonstrations have been carried out, with the mixed reality operation alongside a PC-based, mouse-operated application. Early feedback has suggested that as a method for technical and vocational training (TVET), it is a system that FE colleges would consider as an innovative Technology-enhanced Learning and Teaching (TELT) and be susceptible to its future use both within a classroom and workshop environment. This work builds upon research carried out for the BEACONING program, part of the Horizon 2020 framework.