The Relationship Between Students' Engagement and the Development of Transactive Memory Systems in MUVE: An Experience Report

The use of educational Multi-User Virtual Environments that provide synchronous interaction, interactive and social learning experiences have the potential to increase student engagement. Due to increased social and cognitive presence, the use of such environments can result in greater student engagement when compared to traditional asynchronous learning environments. In this work, we hypothesized that students' engagement in collaborative learning activities will increase if Transactive Memory System constructs are present. Thus, we employed the theory of TMS that emphasizes the importance of Specialization, Coordination and Credibility between members in a team. The results show that there is a significant correlation between the development of TMS and students' engagement.


Introduction
The need for education to become more active, engaging and customized to the learners' individual needs is imperative [1]. One of the ways that higher education institutions are adapting to this change is through the adoption of technology to assist learning and teaching. While modern students are familiar with the use of the Internet, smartphones, high quality 3D graphical computer games, virtual reality and other technological affordances, in their everyday life, it is important to engage them in the learning process when in classroom. Student engagement in higher education is a topic that drew much international research and literature [2], and there are studies suggesting the positive effects of technology [3][4][5] towards student engagement. One of the many technologies that are being used to support students, is the use of Multi-User Virtual Environments (MUVEs). These are computer generated 3D virtual worlds, in which users can interact with and with each other using their Avatar [6]. Employing such environments in education, enables us to develop educational activities that promote collaboration, socialisation and engagement [4,7,8].
Effective communication and collaboration contribute to student learning and are integral parts of the development of Transactive Memory System (TMS). TMS development within working groups has been identified to be very beneficial. The TMS theory relates to the encoding, storage and retrieval of information, and can provide the option to recall previously visited areas of information, and to identify relevant knowledge [9]. It also enables to create awareness of 'who knows what' within a group, and this provides opportunities for effective collaboration. However, while there is a lot of knowledge around the phenomenon of TMS in the field of organisational psychology and behavioural sciences, little is known about the potential relation of TMS with students' engagement and especially, within a MUVE. To investigate this, we have designed a 10-week experimental study, implemented as part of a Computer Science module, delivered through a MUVE we designed. Section 2 presents the theoretical underpinnings of this investigation, and Section 3 describes the methods, instrumentations, and experimental procedures of this study.
The results of this investigation are presented in Section 4, and Section 5 provides a discussion and concludes the paper.

Students' Engagement in MUVE
Students' engagement is an important topic in higher education in general, and Computer Science education in particular. Initially, the concept of student involvement in education was defined as the amount of both physical and psychological energy a student commits on the educational experience [10], and there is a debate about the meaning of student engagement since then [4,11]. Especially in Computer Science there are several ongoing discussions on how academics view the concept of engagement (e.g. [12,13]), and how students' engagement in computer science classes can be improved [14]. This is even more important when we are dealing with first year students in introductory Computer Science modules [15,16].
A number of researchers have defined engagement as the involvement, participation or commitment of the student in learning. Particularly, some suggest that students engagement refers to the extent to which students are involved and actively participate in learning activities [17]. Others indicate that engagement relates to the effort in time and energy the student commits to purposeful educational activities [18]. These definitions enable us to summarize and understand the close relationship between the student and the need to devote effort, time, involvement, active participation and commitment to the learning process. In this work we consider students' engagement by measuring three types of Student Engagement [19,20] and adapting them to the topic of MUVEs: • Behavioral Engagement -what is the student's behavior towards learning through an MUVE e.g. i) I follow the rules of the MUVE; ii) When I am in the MUVE, I just 'act' as if I am learning; iii) I am able to consistently pay attention when I am taking the online class in MUVE.
• Emotional Engagement -How the student is feeling when following the class in the MUVE e.g. i) I like taking the online class in MUVE; ii) I feel happy when taking online class in MUVE; iii) I feel bored by the online class in MUVE.
• Cognitive Engagement -whether the student is following the material he/she read within the MUVE to further develop his/her knowledge e.g. i) I study at home even when I do not have a test; ii) When I read the course materials, I ask myself questions to make sure I understand what it is about; iii) I talk with people outside of school about what I am learning in the MUVE.
The topic of student engagement through the use of technology mediated learning tools and online learning environments has been positively investigated in the literature, and one of the methods that can increase students' engagement is through the use of the educational MUVE [4].
A MUVE is a computer generated and persistent threedimensional environment in which users can navigate and interact with the environment and others, using a virtual representation of themselves known as avatar [6]. MUVEs enable users to immerse in the virtual environment and communicate, interact and coexist in the same shared space at the same time. Over the past few years, the use of MUVEs gained a lot of popularity in education [21] especially in a mixed method teaching and learning scenarios. Educators who are interested to improve, differentiate and enhance their teaching and learning practices use these environments that offer a range of functionalities and possibilities that cannot be found in the traditional online learning systems [22]. MUVEs offer access and synchronous participation in learning activities that make learning more interesting [23], social [24] and engaging [25]. MUVEs allow teachers to develop immersive experiential and problem based learning activities to support and engage students in the learning process [8].
One of the most important attributes of a MUVE is the feeling of presence that the user is developing when interacting and participating in the environment. Presence is concerned with the extent to which the individual feels present in the virtual environment rather than the physical [26], experiencing the illusion of 'being there', regardless if 'there' actually exists in the real world [27]. It has been established that increasing presence also increases learning and performance [28] and it has been suggested that when managed properly, the sense of presence in a MUVE can increase student engagement and improve learning [29,30]. It was also identified that presence is positively correlated with learning success, because it motivates and engages students in learning [31].
Furthermore, due to increased social and cognitive presence, the use of such environments can result in greater student engagement when compared to asynchronous learning environments [32]. Furthermore, the ability of a MUVE to bring students together in the same shared space helps to develop the feel of belonging to a community and this removes the lack of student engagement and isolation that occurs in traditional online environments [33].
Thus, while in traditional face-to-face delivery, instructors experience difficulties maintaining students' engagement and successfully promoting active learning [34], other methods (e.g. [16,35]) including the use of technology as a mean to draw students interest with modern learning methods is a great asset in the toolbox of each academic. Robinson and Hullinger [36] mentions that one of the way educators can offer modules that promote student engagement, is through the use of online learning environments, as these tools are now used at most of the higher education institutions [37]. Such tools are usually used as repositories of information and to provide interactive and collaborative activities [38].

Transactive Memory Systems in MUVE
TMS is defined as a memory system of a group that comprises of the memory systems of individuals, and the processes involved for communicating the information within the group [9]. The processes involved in this procedure are what is needed so the group is able to encode, store and retrieve information held by its members. Following these processes members of a team can facilitate collaboration by utilizing the maximum of their knowledge and information resources [39] and in extend achieve team efficiency and satisfactory results. As a consequence, being part of a team that has developed and utilized TMS can be a positive and fulfilling experience at an individual level [40]. There is a substantial body of research that looked into the development of TMS in several contexts and using different parameters, and TMS proved to be very promising for the functioning of couples, teams and groups in face-to-face [9,41] and online communication [42,43].
A widely used and validated measure for TMS is the one by Lewis [44], where TMS is decomposed into three parameters: Specialization, Credibility, Coordination, and each one is measured by five items in a Likert scale. According to Lewis, specialization deals with unique information and knowledge that a member holds and can be shared or is valuable to the rest of the team as long as the knowledge available is known to, and accessible by others. Coordination is about the ability of the team to work together, coordinate task execution and manage misunderstanding. Finally, credibility is about trusting the knowledge others are sharing and decisions they make.
In addition, measuring TMS is not only a matter of the information and knowledge available, but it is also related to the willingness of the people to contribute and to the task at hand thus, we argue that TMS should be positively related to students' engagement. A successful collaboration should not only be judged by the final output of the team.
Many tools allow the researchers to collect log data of users' collaborative activities and provide the opportunity to mine this data for behavioral and interaction patterns related to collaboration (e.g. [45]

The VirtualSHU
To conduct this investigation, the VirtualSHU MUVE has been used (Fig. 1). The environment is implemented using the Opensimulator MUVE platform, and its design is representing a common educational setting featuring different areas dedicated to each of the topics of the module. In particular, the environment has a main campus with lecture rooms, activity and recreational areas, and a courtyard for students to meet and setup for activities. An orientation area is also provided for students to learn the basic navigation and features of the environment. The environment provides a fantasy and 'sandbox' areas in which students can build their own objects. Lastly, a quiet area is also provided for students who do not wanted to be distracted, or are 'away from keyboard' but do not want to disconnect from the environment. Information on how someone can visit the VirtualSHU can be found here [50]

Experimental Study
To conduct this investigation, we have setup an experiment for a period of 10 weeks to support the tutorial sessions of the Introduction to Information Communication Technology (ICT) module at our university. The module consisted of four tutorial classes with 16 to 20 students attending each class. Students were put in groups of 4 and 5 in the beginning of the academic year, and each session lasted for 60 minutes. During the period of the module, students were collaborating through the MUVE in completing module related tasks (Table 1). All tutorial tasks, for the duration of the experiment, took place within the virtual environment. To access the environment, students had a computer at their disposal, and they were accessing the visual aspect of the environment using their computer monitor, as well as their keyboard and mouse to interact with other users and the environment. The communication affordances of the environment (nearby chat, instant message, group message) were utilized to establish communication and information sharing among students and tutors. Each week's topic was taking place in different dedicated rooms, utilizing a range of inworld tools designed to provide access to PowerPoint slides, website loaders, YouTube videos and information boards.
To design the learning activities, we have used McGrath's typology of tasks [48]. This is a validated, and established taxonomy illustrating activities that need to be performed at each stage of the group's development. We have used this taxonomy when designing activities to keep them diverse and ensure that results would not be activity dependent, to develop and maintain the interest and motivation of students, and to allow them to develop different skills through each activity according to the learning goals of the module. The designed activities required students to generate ideas, perform action tasks, make decisions, solve problems, plan, and resolve conflicts of viewpoints. Furthermore, while having a range of activities, students had the chance to use multiple tools within MUVE for communication, task execution and information sharing, ensuring that results will not be dependent on a single communication tool, and to allow the dynamics of the team to be explored at the maximum possible. Students were assigned a topic of research, and created a shared cloud document for notetaking, and preparing a presentation for the next session (Activity 6) Planning Week6 Students have spend some time finishing off their notes and present them in class..

Week7
Groups were assigned a topic, and students had to review in-world information, and prepare for a discussion (Activity 8).

Week8
An in-world and classroom discussion on the use of IoT in everyday life. Students presented their viewpoints on the topic.

Data Collection Instruments
From the overall 71 students who were enrolled on the module, 48 students (34 male and 14 female), between 19 and 23 years old participated in the data collection.
The instruments used in this study were adapted from existing validated scales. To measure students' engagement in the MUVE and address RQ1, we have adapted the Engagement Scale initially developed by Fredricks et al. [20], and further edited by Sun and Rueda [19], measuring three types of Student Engagement: Behavioral, Emotional and Cognitive Engagement. Initially, the scale was designed to measure children's levels of school engagement, and Sun and Rueda [19] had modified some of the items to measure engagement of graduate and undergraduate students in distance education settings. To adopt the scale and ensure its fitness to the purpose of our investigation, we have further modified the scale by removing two items relating to homework and revisiting recorded lectures (BE5 and CE7 from the Sun and Rueda [19] scale), as these were not applicable to our case. Due to this change, it was imperative to ensure the reliability of the questionnaire, therefore we have calculated an internal consistency coefficient test (Cronbach's a) and the results indicated high internal consistency among the items comprising each scale.

Figure 2: Examples of Students Collaborating in the MUVE
To investigate the development of a TMS among groups working within the virtual world and address RQ2, we adopted the Transactive Memory System scale developed by Lewis [44]. The scale investigates the factors of Specialization, Credibility and Coordination and its interpretation suggests that when a TMS exists, it causes specialized knowledge, trust in each other's knowledge, and coordination in tasks processing.
While the reliability of the instruments used in this study was previously validated and reported, we have also performed reliability tests on the scales using the Cronbach's alpha coefficient. The results were consistent with the index reported by the original authors of each instrument; suggesting high reliability and internal consistencies among the items comprising each scale. All scales were measured using a 5-point Likert rating (5 = strongly agree, 4 = agree, 3 = neither agree nor disagree, 2 = disagree and 1 = strongly disagree). To collect the results, an online questionnaire was administered to students at the end of the experiment (available at [51]).
In addition to the quantitative data collected, we have employed observations and informal enquiries to collect additional data. This included classroom observations by tutors and informal discussions with students to better understand their behavior, performance, and environment use. The experiment was initiated with the assumption that the students had limited social interaction with each other, due to the fact that they were 1 st year students who were only recently introduced to each other during the course induction week. The students were not expected to have specialization in any specific area, credibility, or any group coordination skills; attributes comprising the TMS phenomenon.

Results
Prior to conducting any statistical analyses, the data has been tested and verified for normality; therefore, parametric tests have been used. TMS results have first been investigated using descriptive statistics and are summarized in Table 2. It can be observed that the factors comprising TMS development have been perceived positively, suggesting that a TMS has been established among the working groups especially for Credibility and Coordination. These results are aligned with previous work [7,46,47,49] validating the effectiveness of the MUVE we designed to support collaboration and TMS development. The results indicate that there was overall medium to high Specialization among the students within the working groups, indicating that the students acknowledged and utilized the unique skills that different group members possessed. Mean Credibility is 4.1 out of 5 meaning that most of the students felt that the information shared, and actions taken during task execution by other members were credible and useful for the task at hand. Coordination construct measured how well the groups coordinate their actions, whether they needed to backtrack a lot and overall how well the coordination of activities achieved. Students in our sample thought that the coordination among their groups was good and that helped them achieve the output of each task. Thus, for us the above results confirm that the TMS was developed and the team members acknowledged the value of each member towards the achievement of the tasks' goals.
The Student Engagement results were then investigated, with respect to RQ1, to extract the students' perceptions of their Behavioral, Emotional and Cognitive Engagement when participating in learning activities within the MUVE and are summarized in Table 3. The results indicated that the students' Behavioral, Emotional and Cognitive Engagement were moderately high during activities. Students' indicated positive responses regarding their overall engagement with the course through the MUVE. For example, 75% of the students indicated that they constantly pay attention when they are taking the class in the MUVE; 58.3% mentioned they like taking the online class in the MUVE; 62.5% they feel excited by their work in the MUVE and 83.3% they mentioned they talk with people outside of school about what they are learning in the MUVE.
We understand that the environment itself might have affected how engaged students were with the material within the MUVE. A Pearson correlation test showed that the affordances, tools and design of the environment correlate significantly (r=0.771, p=0.000) with the student engagement scale. 66.7% rated the environment design was stimulating; 66.6%, thought that the object metaphors were intuitive; 75% mentioned that the amount of information displayed on the screen was adequate.  Specifically, the tests revealed positive correlation between Specialization and Cognitive Engagement (r=.56, p=0.000) indicating that the specialized knowledge and/or skills that members possessed relate to the cognitive engagement of the students. Credibility positively correlates to Behavioral (r=.51, p=0.000) and Emotional Engagement (r=.42, p=0.003) indicating that the development of credibility relates to how much involved is a student in terms of the behavior and emotional engagement to the course. Furthermore, positive correlation was also identified among Coordination with Behavioral (r=.53, p=0.000), and moderately positive correlation for Emotional (r=.38, p=0.007) and Cognitive Engagement (r=.32, p=0.029). This is an expected outcome since in order to coordinate actions within group activities students need to be highly engaged to the task.

Observations
One of the most interesting observations identified was that students were engaging in activities that required the team to delegate tasks in pairs or individuals. Tutors observed that activities requiring input from all students for a successful completion, was making students to contribute in order to make sure that they satisfy the rest of the team. This observation was later validated through the students' responses in the TMS questionnaire.
Another interesting behaviour observed by tutors was that the longer students were involved in the activity, the more they seemed to engage and be productive. It was also identified that Session 1D: Pedagogy ITiCSE '19, July 15-17, 2019, Aberdeen, Scotland, UK students responded and delivered better and more complete results in less complicated tasks. Informal discussions between students and the teaching team, revealed that when a reward was offered upon task completion, students were engaging and persisting in trying to complete the activity. Tutors observed that the more the students were required to interact with each other during tasks, the more they kept engaged in the learning activities. This included not only knowledge transfer but also social interaction. Furthermore, since the student interactions established during task execution, active students appear to attract the attention of the disengaged students. Some student groups were informally enquired about their group work at the end of each session, after the fourth week of the experiment. Students suggested that as time passed their collaboration evolved, they were becoming more confident in coordinating, communicating, delegating activities based on the skills of the individual team members, and trusted each other to complete their assigned tasks. These is in line the TMS results.
The students experience with the MUVE was also inquired and their responses were very positive towards the use of the environment to support the module. Students appreciated the immersive feeling of being in the virtual environment and the ability to collaborate and communicate with others in a technology enhanced learning setting. Students indicated that the environment was a useful addition to learning and that the learning experiences were enjoyable. The students also agreed that the use of the environment as an alternative method of learning and undertaking activities was very engaging, as it was different to the traditional sessions they are used to. Some students further discussed the ability of the environment to promote socialization by reducing awkwardness and shyness in social interactions.
However, some students indicated that the 3D graphical and 'gaming like' element of the environment was found as a distraction. Some students provided examples of students disengaging from the activity when finishing their assigned tasks early, and start editing their avatar, playing around or even disturbing other team members by virtually pushing, or spamming the chat room.

Discussion
The importance of promoting and accommodating for student engagement is crucial to ensure quality of learning. An effective modern classroom requires the use of technology, to establish effective collaboration among students, and utilization of learning activities that engage, stimulate and maintain the interest of students. The results of this study suggest that a MUVE can successfully engage students during collaborative learning tasks, through the building of TMS within groups. We have identified that when collaborative activities are designed within an MUVE, these can support Emotional, Behavioural, and Cognitive Engagement (RQ1), since Credibility, Specialisation and Coordination are achieved during the collaborative tasks. We have also determined that levels of Students' Engagement in a MUVE are significantly correlated with the factors that contribute to the development of a TMS within the environment (RQ2). In addition, as time passes, students are getting more familiar and develop better working relationships with their peers in the group and becoming more effective and productive.
Based on the results, observation, and lessons learned during this experimental project, some suggestions and considerations for the design of educational activities within a MUVE have been devised. Firstly, it is important to design activities that require equal input from all students, both in effort and time required for completion. This is necessary in order to ensure that all students are required to engage with the learning content for a successful outcome. Furthermore, it is important to ensure that the learning activities include elements that will involve all students of the group, to allow students time to focus and engage, as well as get to know their peers, each other abilities and capabilities. However, it is also important that these activities are intellectually stimulating and challenging to ensure that students are not losing interest.
Another aspect that an educator could consider is the social aspect of the activities which is essential in the learning process but also for the development of TMS. To promote social connections among students in the MUVE and in classroom, the activities design should aim to facilitate socialisation and let the students showcase their unique skills, knowledge and abilities so they can feel valuable to the group and valued by their peers. This can promote Cognitive and Emotional Engagement. This can be achieved by ensuring that a discussion, brainstorming, and/or information sharing is established within the group preferably at the beginning of the collaboration [7,24]. Finally, the successful completion of activities should require students' attendance and participation in the MUVE, activities should require students to reflect on their learning in order to understand what they have been taught.
While this work has provided some very interesting insights on the topics under investigation, there are some limitations that may have impacted the results. One of the most important limitations of this research project is that students were also conducting activities offline as part of their normal university life, and this had an impact in their communication channels. However, this was a conscious decision as the module team wanted to ensure that the students would communicate and collaborate in real life as well as the virtual world, without being isolated during the semester. Future work will focus on collecting, further data in qualitative form that may help to develop a better understanding of the relationship between Engagement and TMS.